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FOREWORD 


The  present  report  Includes  a  number  of  papers 
centered  around  educational  technology  which  were  presented  at 
a  conference  at  the  Bromwooas  Residential  Center  of  Washington 
University. 

This  conference  was  sponsored  by  the  Department  of 

.  * 

Psychology  of  Washington  University  under  Office  of  Naval  Research 
Contract  Nonr  8l6(l4)  and  by  the  Department  of  Psychology  and  the 
Office  of  Research,  School  of  Continuing  Education  under  U.  S. 

Office  of  Education  Contract  No.  OEC  3-7-061134 -1532,  Research 
Project  6-1134.  A  full  report  of  this  conference  is  available 
as  a  research  publication  of  the  School  of  Continuing  Education 
of  Washington  University  as  "Psychological  Research  in  Classroom 
Learning,"  edited  by  King  M.  Wientge,  Philip  H.  DuBois,  and 
Harry  Gaffney. 

The  present  report  includes  papers  of  special  interest 
to  research  workers  in  the  field  of  technical  training  as 
exemplified  in  the  program  in  this  area  carried  out  at  Navy,  Memphis. 

Acknowledgment  is  made  of  the  assistance  of 
Dr.  King  M.  Wientge  and  Dr.  Harry  Gaffney  in  editing  the  earlier 
version  of  this  report  and  of  the  personnel  of  the  School  of 
Continuining  Education  who  riade  the  arrangements  for  the  successful 
meeting  at  Bromwoods:  Dean  Lynn  W.  Eley,  Dean  John  B.  Ervin,  and 
Mr.  Andrew  Thomas. 
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A  SYSTEMS  APPROACH  TO  ADULT  LEA RUING 


Robert  M.  Gagne 


Can  one  apply  the  methods  of  systems  design  and  develop- 
sent  to  adult  learning?  Yes,  indeed  one  can.  In  fact,  the 
current  situation  and  urgent  requirement  for  a  comprehensive 
course  of  action  in  this  field  makes  adult  learning  a  "natural" 
for  the  application  of  systems  design  methods. 

Let  me  be  clear,  first  of  all,  that  I  am  talking  about 
systems  design  and  development,  and  not  about  systems  analysis, 
which  is  a  much  more  specialized  technique.  One  can,  it  is 
true,  make  an  analysis  of  the  current  situation  pertaining  to 
the  learning  and  continuing  education  of  adults,  using  techniques 
of  operations  analysis  or  the  even  more  specialized  techniques 
of  cost  effectiveness.  But  although  the  results  of  such  studies 
might  well  contribute  some  valuable  information,  it  would  be  my 
contention  that  the  considerably  broader  point  of  view  of  systems 
design  is  needed.  In  simple  terms,  we  can  see  in  broad  outline 
what  is  wrong,  and  what  is  missing.  What  we  need  is  a  thorough 
and  systematic  effort  to  design  a  system  which  meets  the  needs 
of  adults  far  better  than  what  we  now  have.  In  other  words,  we 
need  to  put  together  all  of  the  technology  we  can  identify,  use 
all  of  the  theoretical  predictions  about  adult  learning  we  can 
make,  and  attempt  to  design  a  new  system  which  will  approach 
optimal  effectiveness. 

The  design  of  a  ^stem  begins  with  a  statement  oLpurpose. 

The  system  is  designed  to  accomplish  something,  a  goal  which 
is  agreed  upon  by  the  society  within  which  the  system  is  to 
work,  or  by  some  segment  of  it.  Such  a  purpose,  or  perhaps  a 
set  of  purposes,  is  rationally  analyzed  into  certain  subordinate 
goals  which  cover  the  various  forms  of  operation  the  system  is 
to  have.  (If  there  is  only  a  single  form  of  operation,  well 
and  good,  but  there  are  more  likely  to  be  several.)  At  about 
this  point,  one  begins  to  talk  about  the  functions  of  certain 
parts  of  the  system,  which  may  be  called  components,  or  if 
they  are  larger  entities,  subsystems.  Also,  at  about  this 
point,  one  has  to  introduce  concepts  like  input  and  output ,  to 
refer  to  the  actions  of  these  components.  A  system  is  organized, 
after  all,,  by  planning  and  bringing  about  some  sort  of  compati¬ 
bility,  or  "match"  between  the  output  of  one  component  and  the 
input  of  the  next  in  a  sequence  of  operations.  In  a  rocket 
system,  for  example,  a  match  must  be  made  between  some  planned 
output  of  a  "first  stage"  of  propulsion  and  an  input  to  the 
"second  stage",  in  order  that  the  latter  stage  can  be  put  into 


operation.  Or,  to  take  a  human  example,  a  "match"  must  be 
designed  betv/een  the  numbers  of  people  using  a  mass  transit 
system  at  any  given  time,  and  the  capacity  of  the  vehicles 
which  transport  these  people.  Input-output  considerations 
of  this  sort  not  only  clarify  what  is  meant  by  the  function 
of  any  component,  but  also  lead  to  decisions  generally  called 
allocation  of  functions  to  different  kinds  of  components. 

I  mention  these  things  mainly  to  emphasize  that  the  design 
of  systems  is  subject  to  engineering  logic.  This  is  not  the 
same  as  scientific  logic,  for  a  scientist  is  not  ordinarily 
concerned  with  purposes  and  functions.  But  these  are  the 
kinds  of  decisions  an  engineer  is  supposed  to  make,  and  is 
used  to  making.  Nor  do  I  mean  to  imply  that  because  engineer¬ 
ing  logic  is  used,  one  must  be  concerned  primarily  with  hard¬ 
ware.  There  are  many  systems  and  subsystems  which  are  composed 
primarily  of  people,  rather  than  hardware — for  example,  the 
system  of  student  counseling  in  schools,  or  the  system  of 
trading  in  stocks  in  a  "market."  It  is  engineering  logic  which 
has  designed  such  systems — a  logic  which  deals  directly  with 
goals,  purposes,  objectives,  functions,  inputs  and  outputs. 

One  can,  of  course,  substitute  another  language  if  this  one 
seems  to  specialized.  But  the  logic  will,  I  believe,  be  quite 
the  same. 

Can  one,  then,  apply  such  engineering  logic  to  the  design 
of  a  system  of  adult  learning?  It  will  be  my  purpose  here  to 
show  that  this  can  be  done,  and  to  describe  the  implications 
this  has  for  action.  But  before  I  launch  directly  upon  that 
task,  it  may  be  well  to  review  briefly  where  we  now  stand  with 
respect  to  adult  learning,  and  what  are  some  of  the  reasons  for 
believing  we  need  a  new  system  at  all. 

The  Problem  of  Adult  Learning 


Learning  occurs  throughout  life.  The  increasing  complexity 
of  the  society  in  which  we  live  and  the  accelerating  rate  of 
social  and  technical  change  are  increasing  the  importance  of 
learning  later  in  life.  The  rate  of  technological  and  social 
change  soon  renders  obsolete  some  of  the  skills  and  knowledge 
which  have  been  achieved  by  the  end  of  formal  full-time  educa¬ 
tion.  But  it  is  not  easy  to  keep  learning  in  all  of  the  appro¬ 
priate  areas  of  later  life.  Gardner  (1963,  pp  8-9)  in  speaking 
of  the  commencement  lecturer’s  adjuration  to  "keep  on  learning" 
says : 

"Unfortunately  the  commencement  speakers  never  tell  us 
why  their  advice  to  keep  on  learning  is  so  hard  to  follow. 

The  people  interested  in  adult  education  have  struggled 
heroically  to  increase  the  opportunities  for  self-development , 
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and  they  have  succeeded  marvelously*  Now  they  had 
better  turn  to  the  thing  that  is  really  blocking  self- 
developnent--the  individual's  own  intricately  designed, 
self-constructed  prison,  or  to  put  it  another  way,  the 
individual's  capacity  for  self-renewal." 

Not  only  is  it  difficult  to  maintain  a  capacity  for 
self-renewal;  the  increasingly  complex  roles  of  citizen  and 
worker  demand  more  than  casual  or  incidental  new  knowledge. 
Organized  systems  of  knowledge  are  achieving  a  place  of  im¬ 
portance  for  all  elements  of  our  society  which  has  never 
before  been  experienced. 

The  importance'  of  adult  education  to  our  society  is  likely 
to  be  enhanced  in  still  another  way  by  increasing  trends  in 
technology  and  automation.  This  is  the  need  for  the  individual 
to  build  intellectual  bridges  between  humanistic  values,  on 
the  one  hand,  and  the  methods  and  accomplishments  of  technol¬ 
ogy  on  the  other.  Liveright  (1964)  sees  the  filling  of  such  . 
a  gap  as  one  of  the  major  purposes  of  continuing  education. 
Technologists,  he  believes,  must  be  exposed  to  opportunities 
for  humanistic  studies  and  exploration,  while  humanists  at  the 
same  time  seek  a  more  profound  understanding  of  the  values  and 
goals  of  technology.  For  those  people  who  consider  themselves 
to  have  finished  regular  school  attendance,  the  bringing  to¬ 
gether  of  the  "two  cultures"  may  have  a  profound  effect  on 
successful  living  in  today's  world. 

Some  Historical  Points  about  Adult  Education 

Despite  its  increasing  importance,  the  direction  for 
adult  education  is  not  necessarily  clear.  Many  types  of  adult 
education  have  been  developed  to  meet  a  variety  of  perceived 
learning  needs.  They  have  been  sponsored  by  universities, 
clubs,  churches,  unions,  libraries,  museums,  private  schools, 
government  agencies,  and  public  schools  (Hendrickson,  1960). 
They  have  varied  from  occasional  lectures  to  involved  sequences 
of  courses  leading  to  diplomas;  from  intensely  vocational 
orientation  to  pure  learning-f or-learning * s  sake;  from  highly 
organized  curricula  to  loosely  defined  discussions;  and  from 
literacy  training  to  the  theory  of  relativity. 

It  is  also  true  that  programs  of  adult  education  have 
served  various  purposes  at  varioas  times  and  places.  On  some 
occasions,  for  example,  such  programs  were  undertaken  largely 
for  "citizenship"  purposes;  this  was  particularly  true  back 
in  the  early  part  of  this  century,  in  those  communities  which 
received  many  immigrants.  At  other  times,  the  emphasis  has 
been  largely  "occupational,"  with  the  attempt  to  provide  a 
means  for  the  working  man  or  white-collar  worker  to  acquire 
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additional  skills  in  his  job  or  profession,  and  thus  to  ad¬ 
vance  himself  within  that  occupation.  There  are  many  offerings 
in  adult  education  today  which  serve  this  purpose.  Still  a 
third  goal  for  the  education  of  adults,  and  one  that  has  con¬ 
tinued  for  many  years,  may  be  identified  as  "self-fulfillment," 
which  is  of  course  only  a  small  part  of  what  Gardner  means  by 
self-renewal.  Many  offerings  in  this  field  are  concerned  with 
arts  and  crafts,  including  such  courses  as  how  to  weave,  design  of 
pottery,  as  well  as  appreciation  of  music  and  art. 

How  adequately  these  courses  have  met  a  real  need  is,  in 
general,  unknown.  The  agencies  which  offer  courses  must  satisfy 
their  customers,  the  students,  at  least  to  the  point  of  attrac¬ 
ting  enough  to  make  continuation  seem  worthwhile.  On  the  other 
hand,  many  agencies  would  reject  a  "customer  is  always  right" 
philosophy,  on  the  grounds  that  the  educational  institution 
should  be  striving  to  lead  its  students  toward  higher  goals 
rather  than  merely  responding  to  the  goals  which  the  students 
are  able  to  express.  In  any  case,  choices  must  be  made  in 
developing  the  curriculum,  and  the  institution  must  find  ways 
of  making  these  choices  rationally  rather  than  accidentally. 

Ho  really  systematic  basis  currently  exists  for  selecting  adult 
education  objectives  or  for  developing  curricula  once  they  are 
defined  (Luke,  et  al,  1959;  Sillars,  1958). 

Superintendents  of  public  schools  in  large  majority  sub¬ 
scribe  to  the  idea  that  adult  education  should  fulfill  multiple 
purposes  (National  Education  Association,  1964).  Perhaps  partly 
as  a  result  of  these  broad  aims,  however,  it  is  recognized  that 
the  curriculum  in  adult  education  often  falls  short  of  meeting 
the  needs  of  adults.  For  example,  a  Commission  of  the  Profes¬ 
sors  of  Adult  Education  has  said  (1961): 

"To  its  great  disadvantage,  the  curriculum  of  adult 
education  has  no  plan.  It  is  largely  an  ^  la  carte 
menu  comprised  of  miscellaneous  items.  It  is  not 
organized  in  a  way  that  provides  continuity  and  inte¬ 
gration  of  learning.  It  is  need-meeting  but  not  goal- 
fulfilling." 

The  Systems  Approach 

V 

It  is  quite  evident  that  the  application  of  a  systems  ap¬ 
proach  to  adult  learning  must  begin  with  a  study  to  determine 
goals.  This  is  necessary  because  it  is  from  a  statement  of 
goals  of  the  system  that  all  the  logic  of  this  approach  flows. 

One  could,  of  course,  have  these  goals  determined  by  a  panel 
of  "experts,"  and  this  would  not ' be  an  entirely  unreasonable 
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approach.  But  insofar  as  a  program  of  adult  learning  is  to 
be  instituted  by  a  community  or  other  segment  of  our  society, 
it  seems  a  wise  thing  to  do  to  attempt  to  determine  what  the 
needs  of  adults  in  this  segment  of  society  are. 

However,  it  should  be  noted  that  I  do  not  think  such  an 
effort  can  be  done  by  means  of  the  more  superficial  "survey" 
techniques.  Rather  a  highly  sophisticated  line  of  questioning 
should  be  followed,  in  order  to  come  as  close  as  possible  to 
"true  needs."  It  seems  to  me  that  this  simply  cannot  be  done 
by  asking  questions  like  the  following: 

"What  courses  would  you  like  to  take  in  adult  education? 
_  Pottery 

-  '  American  Civilization 

Modern  Poetry 

_  Stenography 

■  Woodworking" 


In  order  to  get  closer  to  true  needs,  one  would  have  to 
ask  questions  based  upon  the  premise  that  learning  means  a 
change  in  capability.  From  this  basic  assumption,  one  can 
then  proceed  to  formulate  the  kinds  of  questions  that  are 
designed  tc  reveal  concrete  instances  of  desire  (on  the  part 
of  those  questioned)  for  changing  their  capabilities.  Only 
when  one  had  collected  a  sufficient  number  of  these  concrete 
instances  of  felt  needs  would  one  be  justified  in  forming  them 
into  rough  descriptive  categories.  Such  categories  might  be 
vastly  different  from  the  names  of  courses  one  now  finds  offered. 
In  this  process  of  determining  the  goals  of  adult  learning, 
one  should  not  impose  pre-existing,  and  possibly  quite  obsolete, 
categories  upon  the  results. 

Another  set  of  operations  is  also  needed  in  determining 
goals.  Such  goals,  particularly  in  today's  world  of  rapid 
societal  change,  must  be  forward-looking.  And  results  based 
upon  adults1  concrete  experience  cannot  in  themselves  embody 
this  kind  of  orientation  to  the  future.  One  must,  therefore, 
somehow  take  account  of  the  changes  in  communications  and  media, 
in  computerization  of  work,  and  in  probable  increases  in  leisure 
time,  all  of  which  we  are  constantly  being  reminded  of  by  our 
present  popular  media.  For  this  purpose,  one  needs  some  kinds 
of  predictions  of  things  to  come  and  of  the  trends  which  get 
us  from  here  to  there.  It  is  difficult  to  see  how  such  trends 
are  to  be  described  without  some  wise  "experts."  It  would  be 
nice  not  to  have  to  use  them,  because  their  reliability  is 
suspect,  but  they  seem  to  be  necessary.  We  must,  I  think, 
depend  upon  the  projections  of  broadly  informed  and  intelligent 
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people  to  give  us  estimates  of  how  much  non-occupat ional  time 
we  will  have  in  the  future,  of  how  many  of  our  tasks  will  be 
performed  by  computer  systems,  of  how  our  daily  lives  will  be 
affected  by  changes  in  the  arts  of  communication. 

I  should  judge  that  goals  for  adult  learning  could  best 
be  determined  by  means  of  a  rational  interaction  of  t  ese  two 
sources  of  information.  From  the  past,  we  would  take  the  most 
highly  concrete  and  empirically  based  information  we  could 
f ind--derived  from  answers  to  questions  about  actual  needs. 

From  projections  into  the  future,  we  would  use  the  most  highly 
informed  wisdom  available.  But  it  would  be  a  rational  melding 
of  these  two  kinds  'of  information  which  would  give  us  the  best 
formulation  of  the  broad  goals  of  a  system  of  adult  learning. 

Formulation  of  ^Objectives 

Perhaps  I  have  said  enough  about  goals  to  indicate  in  broad 
outline  what  seems  to  me  a  sensible  approach  to  their  determi¬ 
nation.  At  the  least,  it  will  be  clear  that  I  think  this  is 
an  essential  first  step.  If  we  are  to  design  a  system,  we 
must  know  what  its  purposes  are  to  be,  what  outcomes  are  to 
be  expected  of  it,  in  order  to  proceed  with  system  development. 

The' kind  of  study  I  have  suggested  would  lead,  then,  to 
broad  goals.  But  they  would  still  be  broad.  They  need  then 
to  be  categorized  further  in  terms  o f  the  human  functioning, 
or  the  classes  of  human  performance,  which  make  them  up.  For 
example,  one  would  need  to  develop  categories  of  what  kinds 
of  human  capabilities  are  involved,  for  the  range  of  human 
adults  included,  in  the  broad  goal  of  increased  occupational 
competence.  These  might  include  such  categories  as  communica¬ 
tion  capabilities,  human  relations  capabilities,  numerical  and 
quantitative  capabilities,  and  so  on.  Such  performance  classes 
might  bear  some  relation  to  the  categories  of  "academic"  sub¬ 
ject-matters,  such  as  language  and  mathematics.  But  one  should 
not,  and  need  not,  prejudge  this  issue.  Just  because  these 
traditional  categories  have  grown  up  around  the  education  of 
youth,  and  seem  therefore  to  represent  a  kind  of  stable 
reality,  is  no  reason  to  suppose  that  exactly  the  same  cate¬ 
gories  would  emerge  in  consideration  of  adult  learning  require¬ 
ments.  1 

At  any  rate,  broad  goals  would  need  to  be  broken  down  into 
more  specific  categories  of  objectives.  Let  me  state  that 
learning,  if  that  is  the  purpose  of  the  system,  cannot  be  con¬ 
ceived  as  a  process  without  relating  it  to  observable  perform¬ 
ances.  And  furcher,  if  a  system  is  to  be  designed,  we  must 
design  it  with  the  aim  of  producing  certain  specific  outcomes-, 
and  that  is  simply  another  name  for  objectives. 


System  Design 


•  Having  defined  system  outcomes,  one  can  more  or  less 
readily  perceive  how  the  usual  steps  in  system  design  may 
be  carried  out.  These  are,  in  order,-  as  follows: 

1*  Subsystem  functions  to  be  performed.  One  must  identify 
the  major  parts  of  the  system  (i.c.,  the  subsystems)  which  can 
be  put  together  to  accomplish  the  systems  objectives.  In  all 
probability,  these  would  include  a  subsystem  of  administration, 
one  for  instruction,  one  for  guidance,  and  one  for  evaluation. 

2.  Allocution. of  functions  within  subsystems.  One  has 

to  plan  how  each  subsystem  will  work,  what  kinds  of  components 
it  will  contain,  and  how  these  functions  will  be  allocated  to 
the  available  components.  In  system  language,  outputs  of 
components  must'  be  "matched,"  or  made  compatible,  with  the 
inputs  of  other  components. 

The  instructional  subsystem,  for  example,  must  involve 
consideration  of  how  the  following  functions  are  to  be  per¬ 
formed:  measuring  entry  capabilities  of  students;  providing 

for  differences  in  rates  or  styles  of  learning;  determining 
proper  sequences  of  learning;  matching  media  with  learning  re¬ 
quirements;  providing  for  retention  and  transfer  of  learning; 
assessing  learning  outcomes.  Obviously,  some  of  these  functions 
can  best  be  done  by  human  beings,  some  by  printed  communica¬ 
tion  media,  some  by  pictorial  media,  and  some  by  various  com¬ 
binations  of  these.  Similar  remarks  could  be  made  concerning 
other  subsystems  involved  in  the  total  system  for  adult 
learning. 

3.  Identifying  components.  At  first  glance,  it  would  ap¬ 
pear  that  a  wealth  of  components  is  available,  and  that  the 
job  is  mainly  one  of  selecting  the  right  ones.  After  all,  we 
have  many  kinds  of  instructional  devices,  including  text¬ 
books,  motion  pictures,  television,  projected  pictures,  and 
many  others.  We  have  many  kinds  of  tests  which  can  be  used 
for  student  assessment  and  evaluation.  And  we  have  much  cate¬ 
gorized  information  pertaining  to  the  guidance  of  the  indivi¬ 
dual  student. 

As  is  often  true  in  system  development,  however,  closer 
examination  of  these  components  may  show  that  many  of  them 
were  designed  for  other  purposes,  and  may  require  either  ex¬ 
tensive  adaptation,  or  actual  redesign,  when  considered  for 
use  of  a  system  to  promote  adult  learning.  For  example,  to 
what  extent  are  textbooks  formed  to  meet  certain  literary 
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"story-telling"  standards,  as  contrasted  with  efficiency  of 
communication,  or  efficiency  of  learning?  To  what  extent 
have  notion  pictures  and  television  programs  been  designed 
specifically  with  artistic  criteria  in  mind,  as  opposed  to 
effectiveness  in  instruction?  Are  slides  designed  as  pretty 
pictures  rather  than  as  instructional  tools?  One  night  even 
ask,  to  what  extent  has  the  oral  communication  of  the  teacher 
been  designed  to  accomplish  a  kind  of  ent ertainnent  task,  as 
opposed  to  the  task  of  specific  behavior  modification? 

In  designing  a  workable  and  effective  system,  questions 
like  these  must  be  faced  squarely,  and  answered  honestly. 
Components  which  will  not  accomplish  the  necessary  functions 
must  either  be  abandoned,  or  modified  so  that  they  will.  This 
aspect  of  system  development  will  not  be  accomplished  well 
unless  it  is  possible  to  ignore  both  sentimentality  and  tradition. 

Plan  for  system  operation.  Having  identified  the  functions 
of  components,  and  selected  or  modified  those  which  will  per¬ 
form  these  functions,  it  is  now  possible  to  put  then  together 
in  such  a  way  as  to  accomplish  in  a  progressive  fashion  the 
goals  intended  for  the  system.  Sometimes  a  flow-chart  is  used 
for  this  purpose.  Let  me  instead  attempt  to  describe  system 
operation  in  a  series  of  steps,  as  follows: 

Step  1.  Adult  student  appears  at  Adult  Learning  Center 
evincing  interest.  Records  of  vital  statistics  are 
made . 

Step  2.  Student  is  given  an  initial  guidance  interview, 
in  which  information  about  1  is  interests,  life  goals, 
previous  education,  etc,,  is  obtained,  and  he  is  given 
general  information  orienting  him  to  the  system. 

Step  3.  On  the  basis  of  information  obtained  in  Step  2, 
student  is  given  a  set  of  tests  designed  to  measure 
his  level  of  achievement  in  certain  broadly  defined 
areas  of  human  competence.  For  example,  what  mathe¬ 
matics  can  he  do,  what  history  can  he  recount  or  in¬ 
terpret,  what  tools  can  ho  use,  what  sort  of  writing 
can  he  perform,  etc,  (It  may  be  noted  that  tests  with 
these  characteristics  probably  do  not  now  exist,  and 
would  have  to  be  developed). 

Step  .  Student  enters  a  second  guidance  interview,  where 
the  results  of  testing  are  available.  V.'hat  is  to  be 
achieved  is  a  match  between  the  student's  g,oals  and  the 
kinds  of  instruction  available. 
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Step  5.  Student  enters  upon  a  program  of  instruction, 
let  us  suppose  in  three  different  areas.  In  each  of 
these,  the  first  thing  cone  is  measurement,  this  time 
to  determine  what  nay  be  called  his  "entering  level" 
of  competence  in  the  specific  area.  On  the  basis  of 
this  test,  the  teacher  (functioning  as  a  manager  of 
instruction)  makes  an  initial  assignment. 

Step  6.  Student  begins  his  study  of  each  area.  Many  re¬ 
sources  are  presumably  available  for  such  learning. 
However,  these  are  not  allocated  at  the  student's  will 
or  prefer* nee.  Rather,  they  have  been  carefully  matched 
to  the  learning  task  to  be  accomplished,  and  to  the  con¬ 
ditions  of  learning  required  for  this  task  (cf.  GagnS, 
1965a;  Briggs,  Canpeau,  Gagn£  and  May,  1966).  Thus, 
pictures  are  used  when  it  is  known  that  learning  will 
be  facilitated  by  their  use  in  place  of  verbal  descrip¬ 
tions  (c.g.,  in  showing  the  operation  of  a  gasoline 
engine);  whereas  verbal  presentations  are  used  when 
they  will  accomplish  a  similar  purpose  (e.g.,  in  repre¬ 
senting  the  questions  asked  by  a  housewife  to  a  tele¬ 
vision  repairman). 

Step  7.  Student  engages  in  a  variety  of  learning  situa¬ 
tions  in  attaining  successive  levels  of  competence. 

All  kinds  of  media  are  employed.  At  frequently  placed 
"stages",  assessment  is  undertaken  to  determine  v/hether 
the  student  has  attained  the  desired  competencies.  If 
he  has,  he  is  given  a  new  assignment  and  sent  on  to  the 
next  stage  in  learning;  if  he  has  not,  his  specific  de¬ 
ficiencies  are  identified,  and  he  is  encouraged  to 
re-study . 

Step  8.  Periodically,  the  student  is  given  an  opportunity 
to  join  with  other  students  in  teams,  seminars,  or  other 
groups  for  the  purpose  of  developing  nev;  ideas  or 
creating  new  products.  The  most  important  critei’ion 
for  joining  such  groups  is  the  prior  attainment  of 
suitable  skills  or  knowledge  relevant  to  the  activity 
to  be  undertaken  by  the  group.  This  is  essential. 
Otherwise,  the  attainments  to  be  aimed  for  by  the  group 
are  not  highly  specified.  They  night  decide,  for  example, 
to  build  an  electric  automobile,  or  to  write  a  book,  or 
to  formulate  a  petition. 

Step  9.  Once  or  twice  a  year,  the  student  is  again  assessed 
by  suitable  testing,  in  order  to  provide  him  with  an  in¬ 
dication  and  a  record  of  his  progress.  Such  assessment 
will  again  be  followed  by  another  guidance  session,  in 
which  goals  may  be  reviewed  and  revised  if  this  is 
found  desirable. 


Returning  nov:  to  the  major  not  of  staf.es  in  system  design 
and  development ,  there  remains  only  one  nore  to  cover: 

System  testing  ( evaluation ) .  This  phase  of  system  design 
should  probably  take  as  its  no del  the  engineering  test,  par¬ 
ticularly  the  kind  called  a  "field  test."  The  question  to  be 
asked  is:  to  what  extent  is  the  system  meeting  the  objectives 
set  for  it?  I  believe  ue  have  the  techniques  to  carry  out 
such  a  test,  although  we  don't  often  use  them.  Evaluation  of 
the  system  probably  needs  to  consider  obtaining  measures  of 
at  least  the  following:  (a)  kinds  and  amounts  of  changes  in 
student  competencies:  (b)  evidence  of  student  motivation  to 
engage  in  continued  learning;  (c)  evidences  of  students'  accom¬ 
plishments,  and  particularly  their  increased  goal-orientation 
and  creative  activity;  and  (d)  evidences  of  realistic  goal¬ 
setting  on  the  part  of  svudents. 

How  that  I  have  given  some  description  of  the  system  as 
a  whole,  perhaps  it  will  1 e  well  to  review  briefly  some  of 
its  characteristics. 

1.  The  system  assumes  adult  motivation  to  enter  into  the 
system.  Such  motivation  may  of  course  be  various,  but  they 
arc  all  enlisted  in  the  service  of  learning. 

2.  The  central  operation  of  the  system  is  increasing 
student  capabilities  (or  otherwise  changing  his  mental  dis¬ 
positions);  the  central  component  of  the  system  is  the  student. 

3.  The  instructional  subsystem  is  designed  to  meet  the 
needs  of  the  individual  student,  and  for  no  other  purpose. 
Learning  is  by  and  for  the  individual. 

4.  Incentives  are  so  arranged  that  initially  varied  motive 
tion  is  channelled  into  a  desire  for  achievement,  a  "love  of 
learning,"  and  social  approval  for  created  products. 

5.  There  is  frequent  assessment,  of  a  sort  and  for  purpose 
that  are  probably  not  met  by  currently  existing  instruments. 

The  emphasis  in  measurement  is  on  "competency  status,"  or 
"what  the  student  can  do." 

V 

C.  A  subsystem  of  guidance,  emphasizing  self-confidence 
and  self-dependence,  performs  integral  and  important  functions. 

7.  System  evaluation  is  planned  from  the  beginning,  and 
is  also  an  integral  portion  of  the  system. 
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Some  Critical  Points 


-  Mow  that  the  systen  as  a  whole  has  been  briefly  described, 

I  can  turn  attention  to  certain  critical  issues  in  its  design, 
and  try  to  point  out  how  they  nay  be  related  to  and  illuiainated 
certain  findings  of  research  on  the  learning  process. 

How  should  a  system  of  adult  learning  differ  from  a 
systen  of  learning  for  children?  Such  a  contrast  may  help 
to  define  the  kinds  of  instructional  objectives  to  be  achieved 
by  adult  learning. 

Basic  Skills.  .A  system  of  learning  for  the  child  surely 
must  take  into  account  the  fact  that  certain  basic  skills 
have  to  be  learned  first.  These  skills,  which  include  reading, 
writing,  and  figuring,  along  with  certain  others,  are  con¬ 
sidered  basic  because  they  are  essential  to  the  process  of 
continued  learning  itself.  In  contrast,  it  would  be  desirable 
if  a  system  of  adult  learning  were  able  to  assume  these  basic 
skills.  However,  the  latter  system  may  have  to  have  special 
"remedial  loops"  for  some  of  them.  If  one  is  interested  in 
the  broad  range  of  the  adult  population,  the  probability  that 
some  people  cannot  read  painlessly,  cannot  add  fractions, 
cannot  speak  standard  English,  and  cannot  compose  a  simple 
written  communication,  has  to  be  faced.  Designing  such  in¬ 
struction  takes  a  good  deal  of  ingenuity.  The  materials  should 
be  designed  to  interest  adults,  mainly,  I  think,  by  representing 
problems  which  are  as  real  to  life  as  can  be  managed.  An  ex¬ 
ample  would  be  the  task,  in  written  composition,  of  "describing 
to  a  novice  how  to  fly  a  kite,"  as  opposed  to  the  task  of  ex¬ 
plaining  "what  flowers  mean  to  me." 

Acquiring  knowledge.  The  system  of  education  for  the 
child  requires  that  he  code,  organize,  and  store  a  great  deal 
of  what  is  called  "knowledge."  These  are  facts  and  rules  which 
have  come  to  us  from  generations  of  scholars  and  investigators, 
which  must  be  made  a  part  of  "what  every  child  knows."  This 
kind  of  knowledge  acquisition  requires  considerable  "mental 
work,"  presumably  of  the  sort  which  keeps  it  organized  and 
relates  it  to  other  knowledge.  Children  (of  all  ages,  in¬ 
cluding  college  students)  continually  seek  ways  of  avoiding 
this  kind  of  mental  work;  that  is  to  say,  they  tend  to  do 
their  thinking  in  terras  of  already  familiar  knowledge  rather 
than  in  terms  which  will  "fix"  new  knowledge.  A  common 
rationalization  of  college  students,  for  example,  is  that 
they  prefer  classes  in  which  they  "discuss  issues"  (using 
their  already  acquired  incomplete  knowledge)  to  courses  of 
instruction  which  require  them  to  "memorize  information  by 
rote"  (which  is  a  highly  inaccurate  way  of  characterizing  the 
acquisition  of  new  knowledge).  This  tendency  to  avoid  "mental 
work,"  however,  can  be  found  in  all  of  us,  and  is  not  confined 
to  children  or  even  to  college  students. 
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The  basic  means  used  to  facilitate  the  acquisition  of  new 
knowledge  by  systems  of  instruction  is  to  give  the  knowledge 
an  externally  imposed  organization  which  will  aid  its  retention. 
This  may  be  considered  one  of  the  major  uses  to  which  scientific 
knowledge  of  learning  and  memory  can  be  put.  It  is  not  possible 
here  to  review  all  of  the  suggestions  which  might  be  made  in 
designing  instruction  for  this  purpose.  I  believe,  however, 
that  special  attention  should  be  paid  to  the  ideas  of  hierarch¬ 
ical  learning  sequences  by  Gagne  (1965a),  or  "advance  organi¬ 
zers"  by  Ausubel  (1960),  and  of  the  use  of  review  questions 
by  Rothkopf  (1965). 

Accordingly,  the  system  for  adult  learning  must  also 
include  a  means  of  external  organization  of  information  to 
promote  retention.  Perhaps  some  adults  will  have  achieved, 
by  this  age,  some  good  efficient  strategies  for  self-learning. 
For  such  fortunate  people,  the  process  of  self-instruction  can 
go  on  with  the  use  of  almost  any  medium-books,  motion  pictures, 
television  programs.  But  many  adults  will  not  have  achieved 
these  strategies,  because  our  school  systems  are  not  noted  for 
encouraging,  their  development.  The  system  must  then  provide 
the  external  organization  to  materials  for  learning  which  will 
make  their  contents  most  readily  remembered.  It  would  seem 
to  me  that  some  of  the  principles  of  programmed  instruction, 
those  that  have  withstood  the  tests  of  time,  could  be  of  great 
usefulness  in  the  design  of  such  materials. 

Productive  thinking.  The  crowning  achievement  of  a  system 
for  adult  learning  is  to  encourage  and  foster  a  variety  of 
productive  thinking  activities  on  the  part  of  the  adults  who 
participate  in  it.  The  first  and  most  important  requirement 
for  such  activities  is  that  they  be  preceded  by  suitable 
mastery  of  relevant  skills,  and  by  the  learning  and  retention 
of  relevant  knowledge.  It  is  simply  not  efficient  planning, 
for  example,  that  a  study  of  public  speaking  can  be  undertaken 
by  individuals  who  cannot  compose  good  English  sentences; 
nor  that  a  discussion  of  local  law  enforcement  can  be  done  by 
individuals  who  do  not  have  knowledge  of  the  structure  and 
function  of  local  government. 

So  long  as  this  critical  requirement  is  met,  the  system 
can  incorporate  a  great  variety  of  creative  and  productive- 
thinking  activities'*  For  example,  there  can  be  groups  devoted 
to  the  development  of  artistic  products  of  all  sorts,  from 
pottery  to  drama;  groups  whose  function  is  to  forunlate  and 
test  out  action  programs  in  civic  and  political  affairs;  and 
other  groups  which  conduct  seminars  seeking  to  gain  new  insights 
and  appreciations  of  literature,  philosophy,  and  history. 

There  would  seen  to  be  no  special  reasons  for  limiting  the 


scope  or  nature  of  problems  to  be  considered  by  well-informed 
adults.  The  key  idea,  again,  is  to  avoid  the  operation  of 
"courses"  which,  owing  to  the  necessity  of  meeting  the  educa¬ 
tional  needs  of  everybody,  regardless  of  his  preparation, 
take  the  form  of  simplified  chats  from  which  virtually  nothing 
can  be  learned. 

Summary  Statement 

In  summary,  then,  it  would  appear  to  me  that  the  engineer¬ 
ing  logic  of  systems  design  and  development  could  be  success¬ 
fully  applied  to  the  design  of  a  system  for  adult  learning. 

1  have  assumed  that  such  a  system  should  serve  the  needs  of 
adults  for  continuing  to  learn,  beyond  the  confines  of  formal 
degree  programs*  The  basic  logic,  however,  would  apply  to 
formal  programs  like  college  instruction  if  one  wished  to 
make  the  kinds  cf  transformations  necessary. 

First,  there  must  be  a  definitive  determination  of  needs, 
which  I  conceive  to  be  possible  by  a  method  which  combines 
the  information  from  adults  themselves  with  projections  of 
future  trends  made  by  knowledgeable  "experts."  From  such  a 
study,  the  purposes  of  the  system  as  a  whole  could  be  defined. 
Once  these  goals  are  stated,  system  design  can  continue  along 
lines  that  are  relatively  well  known,  to  define  subordinate 
objectives,  allocate  functions  to  both  human  and  machine  com¬ 
ponents,  outline  a  plan  for  operation,  and  a  set  of  techniques 
for  field  testing.  In  broad  outline,  I  should  expect  the 
system  to  contain  subsystems  devoted  to  the  functions  of  ad¬ 
ministration,  instruction,  student  guidance,  and  evaluation. 

It  is  notable,  perhaps,  that  the  function  of  guidance  in  such 
a  system  is  quite  as  essential  as  the  functions  of  other  sub¬ 
systems.  It  would  be  a  key  factor  in  avoiding  the  aimlessness  . 
of  current  attempts  at  "adult  education." 

Each  of  these  subsystems  needs  a  careful  design.  I  have 
made  a  few  suggestions  regarding  the  structure  of  an  instruc¬ 
tional  subsystem.  This  subsystem  must  first  of  all  attend  to 
the  need  for  "filling  in"  those  basic  skills,  including  numer¬ 
ical  computation,  tool-using,  oral  and  written  communication, 
that  are  known  to  be  essential  to  the  pursuit  of  learning 
itself.  Secondly,  it  must  insure  the  acquisition  and  retention 
of  many  kinds  of  previously  codified  knowledge.  Selecting 
this  knowledge  is  of  course  a  difficult  problem  for  system 
design  to  face.  But  the  function  should  not  be  avoided  or 
skipped,  on  the  basis  of  any  kind  of  rationalization  (such  as, 
for  example,  that  "knowledge  rapidly  becomes  obsolete"). 

Systematic  knowledge,  whether  it  is  looked  upon  as  "facts," 
"principles,"  or  whatever,  is  essential  to  the  successful  per¬ 
formance  of  the  third  instructional  function  I  have  described, 
namely,  creative  activity  and  productive  thinking.  Such  a 
function,  in  its  many  varieties,  represents  an  ultimate  goal 
for  a  system  of  adult  learning. 
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COMPUTER  ASSISTED  INSTRUCTION  (CAI): 
SOHE  FACTS  AND  FANCIES 

James  J.  Regan 


It  should  be  indicated  at  the  outset  that  the  point 
of  view  from  which  this  paper  was  written  is  that  of  a 
manager.  The  author  has  had  for  the  past  several  years 
a  responsibility,  largely  administrative,  for  a  developing 
program  of  CAI  research  and  before  that  time  a  more  direct 
involvement  in  both  programmed  instruction  and  early  appli¬ 
cations  of  the  .computer  in  the  instructional  process  (al¬ 
though  still  from  the  management  side  of  things).  As  a 
result,  this  discussion  will  take  the  form  of  an  overview. 

The  computer  has  a  role  to  play  in  a  variety  of  per¬ 
sonnel  functions.  It  has  been  used  as  an  automatic  data 
processor  for  a  number  of  these  functions  almost  from  its 
beginnings.  A  general  view  of  the  uses  of  computers  in  the 
personnel  area  may  be  found  in  the  proceedings  of  an  ONR 
symposium  on  automating  personnel  functions  (1965).  The 
present  discussion  will  be  concerned  with  the  role  of  the 
computer  in  a  particular  personnel  function— that  of  educa¬ 
tion.  There  are  a  number  of  uses  to  which  the  computer  can 
be  put  in  furthering  educational  goals.  It  can  keep  ad¬ 
ministrative  track  of  the  students,  simulate  the  environment 
in  which  a  student  is  trained,  serve  as  an  elaborate  aid  in 
calculation,  and  administer  and  score  tests.  However,  its 
potentially  most  important  function  in  education  is  tutorial, 
and  it  is  this  role  that  is  considered  here.  For  the  pur¬ 
pose  of  this  discussion  the  tutorial  role  will  be  referred 
to  as  Computer  Assisted  Instruction  (CAI). 

There  is  no  educational  research  and  development  area 
changing  as  rapidly  as  is  CAI.  For  this  reason  and  because 
the  information  is  available  elsewhere  [e.g.  Hickey  (1967) 
and  Zinn  (1966)]  CAI  status  in  the  newsletter  sense  will 
not  be  exhaustively  dealt  with  here.  .Instead,  a  brief 
selected  description  of  the  status  of  CAI  will  be  presented 
under  the  headings  of  CAI  centers,  languages,  programs, 
support,  and  economics,  A  second  topic  will  be  concerned 
with  status  in  the  more  general  sense  of  "where  is  CAI  and 
where  is  it  going."  A  final  section,  in  deference  to  the 
subject  of  this  conference,  will  be  devoted  to  research 
issues  and  opportunities  in  CAI. 


What  Hakes  up  the  CAI  World 


Selected  oajor  centers  of  R&D  [Hickey  (1967)  lists  20] 

The  Watson  Research  Center  at  Yorktown,  Hew  York,  is 
responsible  for  some  of  the  early  work  in  CAI,  developing 
programs  in  areas  such  as  German  and  statistics.  These 
programs  have  been  implemented  using  an  author  language 
called  C0URSEWK1TER.  CAI  systems  using  IBM  equipment, 
programs,  and  languages  developed  at  Yorktown  have  been 
installed  experimentally  at  a  number  of  universities.  A 
new  CAI  system  (the  1500)  has  been  developed  at  another 
of  IBM*s  locations,  the  one  at  Los  Gatos,  California. 
Dartmouth  College  has  a  CAI  program  using  a  GE  computer 
and  a  student  language  called  BASIC.  By  using  this  very 
simple,  somewhat  restricted  language,  a  library  of  programs 
in  16  categories  has  been  developed.  The  system  at  Dart¬ 
mouth  is  accessed  by  a  network  of  New  England  schools.  The 
Systems  Development  Corporation  at  Santa  Monica,  California, 
using  an  author  language  called  PLAHIT,  has  developed  among 
other  programs  one  in  statistics  and  one  in  counseling. 

Bolt,  Beranek  and  Newman  of  Cambridge,  Massachusetts, 
is  responsible  for  some  very  early  work  in  CAI  which  was 
concerned  with  the  identification  of  non-verbal  sounds. 

They  also  developed  the  SOCRATIC  system  which  has  a  number 
of  special  features  allowing  the  student  to  engage  in  a 
limited  form  of  conversation  with  the  computer.  A  program 
in  medical  diagnosis  has  been  developed  as  has  one  in  which 
the  subject  solves  a  mystery.  The  SOCRATES  program,  ini¬ 
tiated  at  the  University  of  Illinois,  has  been  used  to 
develop  instructional  strategies  which  have  been  tested 
using  academic  course  material.  Also,  at  the  University 
of  Illinois,  the  Coordinated  System  Laboratory  has  developed 
a  large  scale  CAI  system  called  PLATO.  An  emphasis  in  this 
system  has  been  on  improving  the  quality  and  decreasing  the 
cost  of  a  student  station.  Plans  now  include  a  2,000  station 
CAI  center  which  can  deal  simultaneously  with  a  variety  of 
educational  topics.  The  Learning  Research  and  Development 
Center  at  the  University  of  Pittsburgh  includes  a  relatively 
recent  CAI  center.  This  center  is  concerned  with  the  devel¬ 
opment  of  improved  pedagogical  languages  and  student  com¬ 
puter  interfaces.  The  computer  based  laboratory  for  learning 
and  teaching  at  Stanford  University  ic  devoted  to  a  large 
scale  operational  CAI  program  at  one  of  the  local  primary 
school  systems  in  which  reading,  arithmetic,  and  spelling 
are  taught  to  grade  school  children. 


Languages 


To  make  CAI  accessible  to  the  educator  and  to  the  student 
it  is  necessary  to  develop  languages  for  use  with  the  computer 
which  do  not  involve  the  complicated  encoding  and  other  limi¬ 
ting  features  of  conventional  computer  languages*  Languages 
which  are  available  for  the  student  to  use  in  devising  his 
own  programs  or  otherwise  to  assist  him  include  the  TELCOMP 
language  developed  by  Bolt,  Beranek  and  Newman  which  functions 
in  important  ways  as  a  computational  aid.  The  BASIC  language 
at  Dartmouth,  regarded  as  an  algebraic  language,  does  not 
provide  for  constructed  verbal  responses.  Among  the  teacher 
author  languages  are  COURSEWRITER  which  does  provide  for  con¬ 
structed  responses;  AUTHOR,  the  language  used  by  the  University 
of  Illinois  SOCRATES  system;  MENTOR,  a  Bolt,  Beranek,  and  Newman 
so  called  conversational  language;  and  PLANI7,  a  Systems  Devel¬ 
opment  Corporation  CAI  language.  In  PLANIT  it  is  possible  to 
select  from  five  types  of  frames  one  in  which  to  present  the 
material  and,  within  these  constraints,  to  operate  directly 
to  computerize  an  instructional  program. 

Programs 

Hickey  has  identified  some  150  CAI  programs  as  of  March, 
1967.  A  review  of  these  programs  by  subject  matter  reveals 
that  almost  a  third  of  them  are  in  the  mathematics  area.  Of 
the  remainder,  a  quarter  are  in  education  and  in  educationally 
related  topics  and  almost  all  of  the  balance  in  academic 
subjects  such  as  physics,  psychology,  economics,  statistics 
and  chemistry.  The  length  of  these  courses  varies  from  three 
minutes  to  one  semester  courses  meeting  two  hours  a  week.  The 
amount  of  each  of  these  courses  which  is  on  line  with  the  com¬ 
puter  also  varies.  Most  of  the  CAI  programs  have  been  developed 
by  Project  Plato  at  the  University  of  Illinois,  Stanford  Uni¬ 
versity,  and  the  several  universities  using  IBM  Yorktown 
facilities. 

The  impression  one  gets  from  reviewing  the  list  of  CAI 
programs  is  that  most  of  them  are  fragmentary  and  that  they 
cover  a  rather  restricted  area  of  subject  matter.  Restricted 
not  from  the  point  of  view  of  the  university  setting  in  which 
most  of  them  are  generated,  since  they  cover  a  broad  range  of 
college  topics,  but  restricted  from  the  point  of  view  of 
applications  outside  of  the  university.  For  example,  a  variety 
of  skills  are  taught  by  the  services  in  addition  to  information 
acquisition.  These,  generally  not  represented  by  current 
programs,  include  troubleshooting.,  decision  making,  perceptual 
motor,  and  team  skills. 
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Support 


Hith  many  other  technological  developments  in  education* 
the  Military  Services  have  been  in  the  forefront  of  CA1  devel¬ 
opment.  For  example,  the  Office  of  Haval  Research  has  been 
an  important  supporter  of  a  number  of  the  CAI  centers  and  pro¬ 
grams  which  have  already  been  identified  and  a  number  of 
others  which  for  reasons  of  time  have  not  been  included  here. 
ONR  has  participated  in  the  support  of  the  PLATO  project  and 
the  SOCRATES  effort  both  at  the  University  of  Illinois,  the 
CAI  center  developing  at  University  of  Pittsburgh,  the  work 
at  Bolt,  Beranek  and  Newman,  and  recently  some  of  the  work  at 
Harvard  University.  In  addition,  it  has  been  involved  in  a 
smaller  way  with  some  work  being  done  on  computer  assisted 
explanation,  on  the  use  of  computers  in  troubleshooting  at 
the  University  of  Southern  California,  and  on  educational 
strategies  both  at  Stanford,  and  Systems  Development  Corpora¬ 
tion.  Finally,  ONR  has  in  the  planning  stage  a  major  CAI 
development  which  goes  by  the  acronym  SLASH.  SLASH  is  a  large 
scale  development  center  to  include  a  library,  an  assist  de¬ 
partment,  a  department  for  software,  one  for  hardware,  and  a 
section  for  studies.  The  Bureau  of  Naval  Personnel  is  en¬ 
gaged  in  CAI  research  and  development  at  its  Training  Research 
Laboratory  in  San  Diego  and  has  produced  planning  studies  for 
CAI  efforts  both  at  San  Diego  and  Mare  Island,  California. 

It  is  also  developing  a  substantial  computer  based  educational 
center  at  the  U.  S.  Naval  Academy.  The  Army  is  a  somewhat 
more  recent  entry  into  the  CAI  development  arena  but  even  so 
has  programs  under  way  at  HvmRRO,  Fort  Benning  and  Fort  Mon¬ 
mouth.  The  Air  Force  has  been  involved  for  some  time  in 
special  applications  of  CAI  at  its  Decision  Sciences  Labora¬ 
tory,  Bedford,  Massachusetts,  and  more  recently  at  the  Training 
Research  Division,  Wright  Field,  Ohio, 

The  Bureau  of  Research  of  the  Office  of  Education  (HEW) 
has  supported  research  efforts  in  CAI  for  several  years  with 
a  number  of  its  current  projects  going  back  to  1964.  Of  some 
20  projects  presently  receiving  0E  support,  19  are  with  uni¬ 
versities  or  state  education  departments.  Thus,  the  focus 
of  these  projects  is  academic,  being  concerned  with  secondary 
and  university  education.  Major  projects  receiving  OE  support 
include  Fenn  State  University  (CAI  in  technical  education), 
Stanford  University  (reading  and  arithmetic  curriculum  for 
use  by  culturally  deprived  children),  and  Florida  State  Uni¬ 
versity  (Science  curriculum  for  grades  7,  8,  and  9).  The  OE 
program  represents  in  sum  the  largest  investment  of  any  agency 
supporting  research  and  development  in  CAI. 
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Economics 


A  ratio  of  1/40  of  instructional  to  preparation  tine  has 
bser  reported  by  Penn  State.  Other  figures  go  up  to  1/70  or 
1/100*  These  figures  are  somewhat  misleading  since  such  of 
this  preparation  tine  is  involved  in  organizing  the  material* 
This  organizational  time  might  also  be  expected  in  instructional 
programming  (e.g*  traditional  programmed  instruction)  which 
does  not  involve  the  use  of  a  computer*  The  time  spent  in 
coding  or  entering  the  material  into  the  computer,  although 
it  varies,  may  represent  but  a  small  portion  of  the  preparation 
time*  PLANIT  for  example  requires  little  of  the  author  in 
terms  of  coding*  COURSEHRITER  calls  for  somewhat  more.  The 
choice  of  language  (if  one  has  a  choice)  is  importantly  related 
to  the  instructional  strategy*  BASIC  for  example  does  not  per¬ 
mit  any  constructed  responses*  PLANIT  has  five  types  of  frames, 
thus  in  some  sense  limiting  the  strategies  to  be  employed* 
However,  along  with  other  features,  it  offers  an  effective  com¬ 
promise  between  flexibility  in  instructional  strategy  and  ease 
of  use*  In  any  case,  it  is  inaccurate  to  blame  the  computer 
entirely  for  the  preparation  time* 

In  addition,  it  can  be  expected  that  instructional  pro¬ 
gramming  will  be  more  efficient  with  the  use  of  a  computer 
than  it  was  when  programmed  instruction  began*  The  computer 
can  assist  administratively  and  in  on-line  editing  (Berkeley, 
of  Information  International,  Cambridge,  Hass*,  is  working 
on  this  problem  with  ONR  support).  Finally,  these  development 
costs  must  be  amortized  over  users  and  time*  The  preparation 
time  when  looked  at  in  this  light  can  be  assigned  in  a  variety 
of  economically  feasible  ways* 

A  preliminary  draft  of  a  KumRRO  report  by  Kopstein  (1967) 
on  the  economics  of  CAI  contains  some  interesting  comparisons. 
The  report  cautions  the  reader  throughout  concerning  assump¬ 
tions  in  the  data  and  in  calculating  indices.  Given  the  assump¬ 
tions  (and  without  going  into  them  here)  Kopstein  predicts  that 
CAI  will  in  ten  years  time  cost  roughly  1/2  as  much  per  student 
hour  as  traditional  instruction.  The  costs  for  higher  educa¬ 
tion  are  roughly  comparable  now  although  for  small  school, 
primary  and  secondary  education  CAI  costs  are  presently  ten 
times  greater.  All  of  the  above  comparisons  assume  that  in¬ 
structional  effectiveness  is  equivalent— in  many  ways  a  con¬ 
servative  assumption. 

So  much  for  status.  Where  does  CAI  stand  and  where  will 
or  should  it  be  going? 


CAI:  Some  Observations  and  Suggestions 


One  can  get  a  contradictory  feeling  about  progress  in 
CAI  to  date.  On  the  one  hand  it  appears  that  tremendous  steps 
forward  are  being  aade.  Activity  is  certainly  increasing, 
at  least  when  measured  by  publications.  CAI  is  just  over 
ten  years  old.  As  of  January  1966  Hickey  (1967)  had  assembled 
100  docunents  relating  to  CAI.  As  of  October  1966,  140 
additional  documents  had  been  collected.  Every  month  CAI  is 
being  inaugurated  on  a  developmental  basis  in  new  locations, 
let  many  of  the  most  recent  installations  resort  to  programs 
prepared  early  in  the  movement.  The  range  of  subject  matter 
programmed  for  the  computer  has  not  increased  materially. 

The  student,  for  the  most  part,  is  still  using  the  typewriter 
with  all  of  its  limitations  to  interface  the  computer.  None¬ 
theless,  there  is  a  great  deal  going  on  and  much  that  is  new 
and  improved  is  on  the  horizon  (e.g.  inexpensive  and  improved 
student  terminals),  but  little  of  it  is  operationally  evident. 

Vhat  are  the  major  problems  to  be  solved  before  CAI 
becomes  an  operational  reality?  Hitzel  (1966)  lists  five: 
the  hardware-software  gap;  appropriate  measures  for  evaluating 
CAI;  time  required  to  prepare  a  course  of  instruction;  de¬ 
vising  a  "mix"  between  computer  and  instructor  components  in 
courses  cf  instruction;  and  incompatible  computer  systems. 
Kopstein  (1965)  sees  six  reseaxch  areas  which  require  attention: 
(1)  subject  matter  structure,  (2)  instructor-student  informa¬ 
tional  coupling,  (3)  student-instructor  informational  coupling, 
(4)  measurement  of  progress,  (5)  motivation,  (6)  differences. 
Host  of  those  involved  in  CAI  would  provide  similar  lists. 

There  is  one  important  problem  that  is  seldom  explicitly 
stated.  CAI  requires  a  "critical  mass"  of  significant  pro¬ 
portions  in  terms  of  facilities,  funds  and  talent  (and  not 
necessarily  in  that  order).  The  fact  is  that  massive  resources 
are  involved.  It  would  be  impossible  on  the  basis  of  the 
fragmentary  programming  and  instruction  available  to  date 
(albeit  considerable  in  the  sum)  to  satisfy  the  Office  of 
Education  or  anyone  else,  unequivocally  and  in  cost-effective¬ 
ness  terms  that  CAI  is  an  efficient,  economical  educational 
procedure.  Public  and  private  agencies  ai a  cautious  in 
fostering  CAI  development.  On  the  other  hand,  massive  resources 
have  been  martialled  for  an  attempt  to  get  to  the  moon  and  for 
fighting  a  war  on  poverty.  Education  -is  a  key  in  meeting  these 
national  goals.  There  should  therefore  be  a  greater  willingness 
than  there  is  to  finance  a  vigorous  attack  on  what  everyone 
agrees  are  the  mounting  problems  faced  by  the  educational  com¬ 
munity.  Astronomical  increases  in  cost  are  occuring  just  to 
keep  up  while  accelerating  amounts  of  technological  information 
further  complicate  the  problem.  CAI  is  one  potentially  im¬ 
portant  way  out  of  this  difficulty. 
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There  are  several  other  important  problems  to  be  solved 
in  CAI  on  which  more  explicit  agreement  has  been  reached. 

There  are  two  pervasive  characteristics  of  CAI  as  it  exists 
today.  The  first,  recognized  by  almost  everyone,  is  the 
limitless  potential  for  enriching  the  learning  process  repre¬ 
sented  by  computer  based  education.  The  second,  perhaps  lost 
to  those  who  are  directly  involved  in  its  development,  is  the 
presently  primitive  nature  of  the  student-computer  relation¬ 
ship.  An  important  goal  for  the  future  is  the  enrichment  of 
this  relationship.  This  enrichment  can  occur  on  several  levels; 
variety  of  material  available  in  the  CAI  mode  and  the  options 
for  presenting  and  responding  to  the  instructional  material 
can  both  be  increased. 

Another  problem  of  concern  to  Hansen  (1966),  Hitzel  (1966), 
and  others,  deals  with  the  tendency  in  CAI  to  want  to  program 
on  the  computer  every  aspect  of  a  given  block  of  instruction. 
Serious  effort  should  be  given  to  discovering  an  efficient 
f>mixM  among  the  on-line  use  of  the  computer,  traditional  ins¬ 
truction,  and  the  off-line  use  of  the  computer.  In  this  latter 
employment  the  computer,  in  addition  to  keeping  records  of  the 
Student’s  progress,  can  select  material  for  his  review  and  study, 
and  either  print  it  our  directly  or  send  him  to  where  it  might 
be  found. 

A  final  problem  which  will  only  be  identified  here  is 
that  of  author-student  languages  and  instructional  strategy. 

This  is  really  a  dual  problem.  If  one  is  willing  to  restrict 
the  computer  to, for  example,  a  programmed  instruction  type  pre¬ 
sentation,  providing  a  language  with  which  the  author  gets  the 
material  into  the  computer  is  a  relatively  straightforward  matter. 
If,  on  the  other  hand,  complex  strategies  and  complex  materials 
are  to  be  used  the  language  problem  becomes  Monumental  [cf  Spolsky 
(1966)3.  Various  intense  attacks  on  this  problem  are  underway. 

Research  and  CAI 


Since  the  topic  of  this  meeting  is  Research  and  Classroom 
Learning,  it  would  be  appropriate  to  make  some  comments  con¬ 
cerning  research  and  CAI.  From  the  point  of  view  of  educational 
research  the  most  important  feature  of  CAI  is  probably  the  avail¬ 
ability  of  adaptive  instructional  modes.  Adaptivity  as  used  here 
refers  to  the  fact  that  the  computer  can  present  stimulus  material 
to  the  student  as  a  function  not  only  of  his  mrst  recent  response 
but  of  some  patterning  of  his  earlier  responses  as  well  as  on 
the  basis  of  a  host  of  aptitude,  personality,  and  personal  history 
characteristics.  This  individualizing  of  learning  affords  for 
example  an  opportunity  to  study  the  relationship  of  individual 
differences  (in  terms  not  only  of  aptitude  but  of  such  things  as 
learning  style)  and  methods  of  instruction. 
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Gentile  (1967)  has  suggested  that  insufficient  attention 
has  been  directed  during  the  development  of  CAI  to  research 
issues  occasioned  by  the  use  of  the  computer,  and  the  more 
general  issues  associated  with  hutaan  learning.  For  example, 
he  cites  some  misgivings  Cronbach  voiced  over  a  decade  ago 
about  the  advantages  of  individualizing  instruction.  That  is 
to  say  about  methods  of  tailoring  instruction  to  individual 
differences.  Cronbach  suggested  that  we  really  know  very  little 
about  what  differences  are  related  to  what  methods  of  instruc¬ 
tion  and  he  further  suggested  that,  until  we  learn  more,  per¬ 
haps  some  plan  carefully  tailored  to  the  group  derived  mean 
would  be  the  most  effective  way  to  go  about  conducting  a 
learning  session. 

The  questions  and  cautions  raised  by  people  such  as 
Gentile  (1967)  are  real  and  they  are  important.  However, 
there  is  an  implication  in  these  questions  that  a  more  research 
oriented  CAI  effort  should  be  embarked  upon  and  that  the  so 
far  predominate  development  effort  should  be  slowed  down. 
Pursuing  the  development  of  CAI  at  this  stage  is  admittedly 
something  of  an  "act  of  faith."  And  it  is  an  important  act 
of  faith  because  the  costs,  in  various  terms,  incident  on 
pursuing  its  development  are  so  great.  However,  there  is  some¬ 
thing  of  a  dilemma  posed  by  trying  to  increase  research  at  the 
expense  of  development.  No  genuine  assessment  of  the  effects 
of  introducing  the  computer  in  the  educational  process  is 
possible  without  a  significantly  large  amount  of  development. 
Piece-neal  comparisons  of  learning  methods  using  fragmentary 
programs  of  instruction  are  simply  inadequate.  Development 
must  reach  a  point  where  longitudinal  investigations  are  pos¬ 
sible  and  where  the  computer  can  be  used  in  ways  that  realize 
its  potential.  Studies  where  a  complex  strategy  of  learning 
a  complex  task  can  be  evaluated  in  a  setting  that  allows  the 
student  a  range  of  responses  begin  to  exercise  this  potential. 
Some  observations  concerning  CAI  research  features  and  possi¬ 
bilities  follow,  assuming  some  RED  balance  has  been  achieved. 

Hartley  (1966)  reviev7ed  about  a  hundred  and  twelve  studies 
which  purported  to  compare  programmed  and  conventional  instruc¬ 
tion.  Only  six  of  these  112  met  the  following  four  minimum 
criteria:  (1)  program  involved  more  than  five  instructor  hours, 

(2)  there  were  more  than  fifteen  subjects  per  group,  (3)  the 
time  to  complete  the  program  was  reported,  (4)  pre  and  post 
results  were  reported.  Of  the  six  studies  meeting  these  cri¬ 
teria  all  indicated  that  the  program  instruction  was  superior. 
Findings  of  this  sort  suggest  that  the  research  to  be  conducted 
on  CAI  should  meet  some  fairly  general,  roinimably  acceptable 
ground  rules.  These  rules  can  include  those  of  the  sort 
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Hartley  used.  Such  additional  ones  as  the  following  night 
also  be  useful:  (a)  some  indication  of  the  on-line/off- 
line  ratio  in  a  given  course  of  instruct  ion ,  (b)  a  descrip¬ 

tion  of  the  nature  of  the  instructional  strategies  employed. 

CM  seems  to  be  avoiding  the  sort  of  comparative  evalua¬ 
tions  Hartley  reviewed,  with  their  attendant  problems.  There 
were  innumerable  studies,  including  some  early  ones  with  which 
tho  author  was  involved,  in  which  a  given  block  of  material 
or  course  was  taught  via  programmed  instruction  and  compared 
with  the  same  material  presented  conventionally.  As  you  know 
most  of  these  studies  showed  that  the  programmed  material  was 
learned  more  rapidly  and  that  achievement  at  the  end  of  the 
course  was  about  the  same.  Occasional  attempts  at  cost  effec¬ 
tiveness  analysis  indicated  that  programmed  instruction,  to 
be  economically  feasible,  had  to  be  confined  to  those  courses 
in  which  large  numbers  of  students  were  receiving  instruction. 

The  reasons  that  it  seems  desirable  that  CAI  avoid  this 
type  of  comparison  are  several.  For  one  thing,  there  are 
fundamental  questions  of  research  design  strategy  in  compara¬ 
tive  analyses  of  the  sort  described  above.  For  example,  if 
an  experimental  evaluation  shows  "B"  better  than  "A"  the  re¬ 
sult  may  hinge  on  the  fact  that  "A"  stood  lower  in  a  popula¬ 
tion  of  A's  than  nBn  stood  in  a  population  of  B's.  One  might 
then  have  suggested,  as  I  believe  Skinner  did,  that  the  next 
task  was  to  devise  a  better  nAn.  Beyond  this  sort  of  con¬ 
sideration  however,  in  CAI  the  question  arises  as  to  whether 
or  not  achievement  as  measured  by  end-of-dass  test  scores  is 
really  an  appropriate  measure  of  effectiveness  at  all.  Hitzel 
(1966)  suggests  that  such  achievement  tests  are  in  fact  not 
appropriate  since  CAI  in  a  fully  realized  tutorial  mode  is 
truly  individualized  instruction.  Thus,  it  would  seem  more 
appropriate  to  measure  effectiveness  by  how  long  it  took  the 
student  to  arrive  at  some  point  of  mastery  and/or  how  many 
repeats  he  required  to  reach  this  point.  In  any  case,  con¬ 
frontations  between  CAI  and  traditional  instruction  are  not 
at  this  point  the  most  useful  way  to  advance  development. 
Licklider  and  others  have  advised  against  such  confrontations 
on  any  terms  suggesting  that  it  is  too  early  to  evaluate  the 
potential  of  CAI.  An  example  often  used  here  concerns  the 
Wright  Brothers'  aircraft,  which,  if  i.t  had  been  compared 
with  the  Pony  Express,  might  have  lost  the  race.  If  this  loss 
had  been  taken  seriously  there  might  have  been  *no  SST. 

There  are  a  number  of  other  research  opportunities  afforded 
by  CAI  which  time  will  permit  us  to  mention  only  briefly.  The 
U.  S.  Naval  Academy  with  CAI  support  is  concerned  with  using 
the  computer  not  only  in  a  tutorial  mode,  which  we  have 
characterized  as  the  essential  characteristic  of  CAI,  but 
also  in  a  broader  educational  management  sense.  The  Academy 
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will  use  the  computer  to  control  a  variety  of  media  (the 
multi-media  approach)  to  be  used  in  instructional  process. 

Here  the  computer  in  addition  to  functioning  tutorially 
selects  various  other  educational  devices  for  use  during 
portions  of  a  given  course  of  instruction,  assesses  the 
student's  progress  for  the  instructor,  and  decides  when  the 
teacher  himself  should  be  called  in. 

Smallwood  (1966)  has  suggested  that  the  most  important 
role  (one  that  receives  insufficient  attention)  that  the  com¬ 
puter  can  play  in  CAI  is  adaptive— the  capacity  of  the  compu¬ 
ter  to  vary  its  presentation  as  a  function  of  a  variety  of 
information  that  it  has  about  the  student.  Smallwood  further 
suggests  that  there  are  two  kinds  of  adaptivity  that  should 
be  exploited.  The  first  is  the  one  which  has  just  been  de¬ 
fined,  the  second  is  the  capacity  of  the  computer  to  change 
its  rules  over  time  as  it  learns  about  the  effectiveness  of 
previous  adaptive  procedures.  There  is  fragmentary  evidence 
available  to  indicate  that  a  student  will  learn  much  more 
rapidly  once  he  becomes  aware  of  the  fact  that  he  is  in  con¬ 
trol  of  the  learning  situation  —  a  state  of  affairs  rarely 
found  in  a  typical  learning  situation.  However,  in  the  case 
of  CAI,  the  opportunity  does  exist  for  the  student  to  control 
the  rate  at  which  he  is  given  information,  the  kind  of  in¬ 
formation  he  is  given,  and  when  he  shall  be  given  it.  It  is 

quite  possible  once  he  recognizes  these  options - options  for 

example  which  allow  him  to  call  for  material  already  presented 
or  to  speed  or  pass  by  information  he  has  already  mastered— 
that  he  will  acquire  information  a  good  deal  more  rapidly. 

There  is  evidence  in  programmed  instruction  and  of  a  more 
limited  form  in  CAI  to  indicate  that  the  relationship  between 
intellectual  aptitude  and  course  achievement  is  lower  in  these 
methods  of  instruction  than  is  conventionally  the  case.  If 
these  findings  are  substantiated ' it  suggests  a  variety  of 
possibilities  including  the  possibility  that  CAI  involves 
aptitudes  not  ordinarily  measured  and  that  individuals  less 
trainable  by  ordinary  classroom  techniques  might  find  CAI 
presentations  more  effective.  Associated  with  this  finding, 
or  as  an  extension  of  it,  is  the  possibility  provided  by  CAI 
of  assessing  the  consequences  of  various  individual  differences 
including  personality  and  learning  style  differences  for 
teaching  strategies.  This  is  an  area  of  considerable  practical 
as  well  as  theoretical  importance  and  the  last  word  can  hardly 
be  written  on  it  without  a  thorough  investigation  in  a  CAI 
setting.  Finally,  general  observations  might  be  made  con¬ 
cerning  the  effect  on  learning  research  over  a  broad  front 
that  will  result  from  the  availability  of  computers. 
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As  ons  looks  at  the  possibilities  identified  above, it  becomes 
clear  that,  in  addition  to  the  usual  features  of  an  automated 
research  activity  such  as  rapid  data  collection  and  analysis, 
the  feasibility  of  exploring  in  some  detail  the  effects  of  a 
variety  of  variables  on  individuals  as  well  as  groups  is  now 
open  through  CM* 

Conclusion 

There  are  many  reasons  why  educators  and  educational 
research  people  have  misgivings  about  CAI  both  in  its  present 
stage  and  as  it  might  develop.  These  reservations  may  be 
founded  on  a  variety  of  operationally  testable  positions. 
Objections  such  as,  "one  cannot  learn,  retain,  or  transfer 
information,"  or  that  "student  attitudes  are  negative"  can  be 
investigated  and  are  useful  in  the  course  of  evaluating  CAI. 
However,  the  position  frequently  taken  which  has  to  do  with 
the  allegedly  impersonal  and  mechanistic  character  of  CAI 
is  a  value  judgment  which  is  not  operationally  useful.  In 
fact,  rather  than  view  CAI  as  impersonal  it  can  be  said,  as 
Christal  (in  Office  of  Naval  Research  1965)  has  suggested, 
that  one  cannot  afford  not  to  give  students  the  personal  . 
attention  available  to  them  through  the  use  of  computer 
based  educational  systems. 
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COMPUTER  ASSISTED  INSTRUCTION: 
SOME  FACTS  AND  FANCIES 
Discussion  by  Robert  Gagne 


Regan's  paper  is  a  thoughtful  and  informative  account 
of  what  is  going  on  in  this  field.  Having  to  comment  on  it 
has  forced  me  to  think  about  things  that  bother  me  about 
computer  assisted  instruction.  I  have  always  had  nagging 
doubts  about  computer  assisted  instruction— -not  because  I  do 
not  believe  it  is  possible— obviously  it  is— but  because  it 
has  not  seemed  to  me  to  be  terribly  useful.  Jim's  remarks 
have  not  dispelled  these  doubts;  in  fact,  I  think  that  some 
of  the  things  he  mentioned  serve  to  reinforce  them.  And  it 
is  not  because  I  do  not  want  to  accept  computer  assisted  in¬ 
struction,  but  because  I  still  do  not  understand  its  rationale 

Now  let  me  see  if  I  can  catalog  what  I  think  to  be  the 
non-useful  things  about  computer  assisted  instruction.  The 
first  is,  as  I  said  at  a  conference  some  years  ago,  that  it 
really  seems  that  what  the  computer  is  doing  is  page  turning 
and  that  this  is  a  very  expensive  way  to  turn  pages. 

We  are  told  that  the  instructional  programs  to  which  the 
student  is  subjected  are  adaptive*  This  can  mean  several 
things.  For  one  thing,  it  can  mean  that  the  instructional 
program  is  what  some  people  call  gaited  in  the  sense  that  the 
individual  can  skip  over  parts  of  the  program  which  he  may 
already  know.  This  is  quite  a  reasonable  thing.  It  seems 
to  me,  however,  that  in  this  case  too  I  wonder  whether  it  take 
a  computer  to  do  this.  The  other  notion  of  adaptiveness  is 
the  notion  of  learning  styles.  I  still  think  that  this  is, 
at  the  present  time,  largely  a  myth.  That  is  to  say,  it  is 
something  that  people  want  to  believe  in  but  which  they  have 
very  little  evidence  on  which  to  base  such  a  belief.  The 
notions  that  some  people  react  better  to  visual • materials 
and  some  to  auditory  ones  is  an  old  idea  that  has  a  long 
history,  with  very  little  evidence  to  back  it  up.  Cronbach's 
statement  is  that  perhaps  these  kinds  -of  learning  styles  do 
exist,  but  he  could  only  find  about  one  identifiable  kind  of 
learning  style  in  the  literature  and  this  one  pertained  to 
a  kind  of  personality  reaction  of  students  with  a  teacher. 
Therefore,  it  seems  doubtful  that  could  be  represented  in  a 
computer. 
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The  third  point  is  that  this  kind  of  instruction  is 
currently  limited  to  a  particular  variety  of  instruction, 
a  fixed  reaction  mold.  The  variety  of  material  that  is 
available  on  a  random  access  basis  may  be  large.  Neverthe¬ 
less,  it  does  have  fixed  reaction  with  the  student  pressing 
a  button  or  typing  and  the  computer  responding  with  a  pre¬ 
viously  programmed  response.  It  has  been  pointed  out  that 
this  is  really  a  very  primitive  mode  of  instruction  and  it 
may  take  many  years  to  develop  more  sophisticated  modes  such 
as  those  which  can  be  called  tutorial  mode,  where  the  reaction 
to  the- student's  response  is  really  adaptive  to  what  he  needs 
at  the  moment,  or  other  modes  in  which  the  student's  thinking 
is,  in  fact,  encouraged. 

So,  I  think  we  are  really  very  primitive  in  terms  of 
what  can  be  done. 

Fourth,  the  technology  of  reacting  to  student  responses 
is  still  pretty  primitive.  It  seems  to  me  that  there  would 
be  a  real  breakthrough  if  the  computer  had  a  means  of  reac¬ 
ting  to  students'  oral  responding  or  even  his  written  re¬ 
sponding  in  a  way  which  did  not  restrict  him  to  pushing 
buttons.  I  do  not  know  how  far  away  we  are  from  that;  but 
technologically  this  would  seem  to  be  a  complicated  business 
which  we  are  not  approaching  very  fast. 

A  fifth  point  is  that  the  whole  matter  of  the  coordina¬ 
tion  between  the  teacher  and  the  computer,  and  the  identifi¬ 
cation  of  their  roles  and  of  their  relative  functions  is  in 
a  very  primitive  stage.  We  should  think  of  the  computer  as 
freeing  the  teacher  to  do  many  things  that  may  be  more  impor¬ 
tant  than  what  the  computer  can  do.  I  am  not  sure  that  it 
has  showed  signs  of  doing  this.  The  teacher  or  somebody  taking 
his  place  has  to  become  involved  in  a  way  which  simply  supports 
the  operation  that  is  being  carried  out  by  the  computer  as 
opposed  to  being  freed  for  other  things.  I  do  not  need  to 
remind  you  that  there  are  many  important  teacher  functions 
that  apparently  cannot  be  performed  by  the  computer,  such  as 
motivating  the  student,  and  providing  him  with  ideas,  standards, 
and  values.  I  simply  remind  you  that  there  are  educational 
functions  that  presumably  no  one  has  yet  suggested  that  the 
computer  can  do. 

Another  point  — and  this  a  very  important  one  —  is  that 
the  emphasis  tends  to  be  on  the  machine  and  not  on  the  student. 

I  think  it  would  be  a  bad  mistake  if  we  forgot  that  in  any 
educational  system  the  student  is  the  central  focus.  Perhaps 
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we  need  to  reread  Dewey  on  the  child  and  the  curriculum— 

I  consider  this  the  best  thing  that  Dewey  wrote  on  education. 
He  wrote  that  you  do  not  decide  upon  what  to  do  about  educa¬ 
tion  by  examining  the  curriculum,  or  by  structuring  the 
curriculum.  You  look  at  the  child,  you  find  out  how  he 
operates  and  that  is  the  important  question. 

Mow,  taking  all  these  things  together  I  continue  to 
have  doubts  that  what  will  ultimately  result  from  years  of 
work,  and  the  expending  of  enormous  funds,  will  really  be 
very  useful.  I  think  we  must  get  back  to  the  individual  and 
I  do  not  mean  this  in  any  mystical  or  even  humanistic  sense. 
Learning  is  fundamentally  a  highly  idiosyncratic  process. 

Me  can  aid  it  by  imposing  external  organization  upon  it,  but 
we  cannot  make  j.t  occur  that  way.  This  is  the  nature  of  my 
doubt.  Why  must  we  take  this  tack  at  all?  Why  can't  we 
build  into  the  individual  himself  the  capability  of  central 
processing  which  is  fundamentally  the  event  of  learning.  He 
have  an  enormous  variety  of  information  displays  to  depend 
upon.  The  individual  can  read  books,  he  can  look  at  pictures 
he  can  listen  to  recorded  messages.  All  of  these  can  perhaps 
be  improved  for  the  purposes  of  stimulation  or  more  broadly, 
communication.  The  computer  can  surely  be  of  great  aid  in 
bringing  about  ease  of  communication  in  making  available  an 
enormous  variety  of  sources  of  information,  but  none  of  these 
functions  improves  learning  itself— they  only  improve  the 
single  condition  of  instruction  that  I  have  called  the  pre¬ 
sentation  of  stimulant. 

I  believe  that  central  processing  can  and  should  be 
built  into  the  learner.  The  learner  should  learn  to  organize 
his  own  learning,  he  should  learn  strategies  for  selection, 
strategies  for  remembering,  strategies  for  transfer.  In 
summary,  I  think  it  likely  that  the  computer  is  not  going  to 
replace  the  student's  nervous  system  and  therefore,  it  is 
bound  to  have  limited  usefulness  no  matter  how  hard  it  is 
pushed. 
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SIMULATION  APPLIED  TO  EDUCATION' 
Edward  U.  Jones 


Simulation  as  a  training  tool  has  evolved  in  20  years  to 
the  point  now  where  it  is  a  major  and  essential  part  of  the 
aerospace  industry.  It  is  needed  to  support  space,  aircraft, 
missile,  and  command  and  control  systems  so  that  appropriately 
trained  personnel  are  available  to  operate  and  maintain  then 
to  the  performance  levels  assumed  in  specifications  and  design. 

Two  recent  developments  will  markedly  enhance  its  future  utility: 

a.  Space  Experience  -  The  Mercury  and  most  Gemini  Astro¬ 
nauts  were  trained  with  simulation  equipment  alone  to  the  point 
where  they  accomplished  their  first  flights  successfully  (12). 

In  addition,  spacecraft  simulators  were  interrelated  with  ground 
monitoring  and  control  systems  to  form  a  systems  trainer.  Part- 
task  devices,  particularly  for  environmental  and  visual  tasks, 
were  used  to  supplement  the  major  devices.  The  spacecraft  pro¬ 
gram  has  truly  been  a  demarcation  point  for  simulation  since  it 
demonstrated  dramatically  the  potential  validity  of  simulators  (5). 

b.  Commercial  Airlines  -  The  transition  to  jets  and  the 
accelerating  growth  in  operations  has  required  the  airlines  to 
use  flight  simulators  for  training  even  beyond  the  degree  of 
reliance  the  military  has  placed  on  them.  The  SST,  airport  and 
airways  crowding,  and  high  aircraft  operating  costs  have  re¬ 
sulted  in  the  economic  need  to  accomplish  as  much  airline  train¬ 
ing  and  proficiency  measurements  as  possible  in  simulators  (7). 

This  paper  will  examine  how  the  technology  now  used  ex¬ 
tensively  in  the  aerospace  industry  might  be  adapted  to  the 
classroom.  The  question  that  will  be  examined  is  the  appropri¬ 
ateness  of  simulation  for  classroom  training.  My  intent  is  to 
look  more  to  the  future  than  to  the  immediate  situation  because 
such  speculation  allows  liberties  not  always  available  when  one 
considers  specific  equipment  capabilities. 

This  examination  will  be  done  from  the  perspective  of  an 
engineering  ps_  chologist  who  is  both  man  and  equipment  oriented 
and  who  approaches  problems  such  as  this  by  synthesizing  the 
factors  related  to  the  needs  of  the  classroom,  hardware  capabi¬ 
lities,  acceptance  to  the  user,  and  cost.  This  results  in  a 
middle  of  the  road  position  between  that  of  the  research  psychol¬ 
ogist  and  the  engineer/computer  specialist.  We  are  particularly 
well  aware  of  the  problems  of  ’,g^dgeteerin£,, ,  that  is,  equipment 
for  equipment's  sake  without  a  clear  analysis  of  its  possible 
use  based  on  need  and  utility,  a  predicament  I  wish  to  avoid. 
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Characteristics  of  a  Simulator 


Although  a  simulator  can  be  broadly  defined  as  any  rep¬ 
resentation,  model,  ox*  picture,  its  definition  here  Kill  be 
restricted  to  a  relatively  accurate  reproduction  of  a  complex 
aan-nachine-onvironircntal  system  with  which  nan  interacts  in 
an  operational  or  maintenance  function  (5).  It  is  not  open- 
looped  as  is  a  cockup,  but  is  closed-looped  and  appropriately 
reactive  to  can's  inputs. 

It  is  either  part  or  near-whole  depending  on  the  needs 
and  economics  of  the  situation.  It  contains  enough  of  the  es¬ 
sential  elements  to  cake  it  appear  like  the  real  situation. 
Generally,  more  than  one  person  is  trained  at  one  tine,  but 
this  depends  on  the  characteristics  of  the  system  being  dupli¬ 
cated.  Training  emphasis  is  on  procedures,  perceptual-taotor 
performance,  information  processing,  and  decision-making.  En¬ 
vironmental  conditions  such  as  motion  and  zero-gravity  are 
occasionally  added.  The  visual  elements  internal  to  the  task 
(interior  of  the  work  station)  tend  to  be  duplicated  more  than 
exterior  visual  conditions  such  as  airports,  highways,  and 
lunar  surfaces. 

A  characteristic  that  distinguishes  then  from  simple  re¬ 
productions  of  the  system  itself  is  the  capability  to  overlay 
malfunctions  or  out-of-tolerance  conditions  on  the  basic  task. 

These  conditions  tend  to  be  those  rarely  encountered  or-  diffi¬ 
cult  to  reproduce  in  the  actual  system  (aircraft  engine  fire). 
Another  key  characteristic  is  the  capability  to  measure  man's 
performance  for  knowledge  of  results  or  administrative  evalua¬ 
tion.  In  some  cases,  parts  of  a  real  system  may  be  used  in  the 
simulation,  but  these  are  driven  artificially  or  are  related  to 
an  artificial  environment.  For  example,  the  University  of 
California  at  Los  Angeles  has  a  driving  simulator  which  consists 
of  an  actual  automobile  on  rollers  with  an  artificial  visual 
screen  projected  fore  and  aft.  The  military  uses  training 
problem  generators  to  supply  inputs  to  real  systems.  Occasionally, 
there  are  simulators  for  man  himself  such  as  that  which  reproduces 
his  thermal  output  used  for  testing  spacecraft  under  more  realistic 
conditions.  However,  this  paper  will  consider  only  simulators 
with  man-machine  interactions. 

In  summary,  simulators  are  defined  here  as  relatively  exact 
reproduction  of  a  part  of  near-whole  tack  that  generally  dupli¬ 
cates  man's  interface  with  a  complex  machine  or  system.  The  de¬ 
vice  reacts  dynamically  with  the  individual  by  responding  dif¬ 
ferentially  to  his  inputs  and  allows  out-of-tolerance  conditions 
to  be  imposed  upon  the  task.  A  simulator  reproduces  a  specific, 
not  a  generalized  task.  For  example,  a  duplication  of  the  707 
cret/station  would  be  a  simulator,  while  a  generalized  duplication 
of  the  crewstation  for  any  aircraft  would  be  a  trainer.  It  is 
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generally  assumed  that  transfer  of  training  is  achieved  through 
this  duplication.  Although  simulators  ere  usually  thought  of 
as  training  devices,  the  aerospace  industry  uses  then  for  other 
purposes  as  well.  There  can  be  a  hierarchy  of  use  sometimes 
for  the  sane  device  which  cut:  include: 

a.  Research 

b.  Equipment  Design 

c.  Man  and  Equipment  Testing 

d.  Procedure  Development 

e.  Training 

f .  Accident  Investigation 

For  exanple,  a  zero-gravity  crew  transfer  sinulator  used 
to  demonstrate  feasibility  and  for  development  of  the  techniques 
of  going  from  one  portion  of  a  spacecraft  to  another  will  be 
used  later  as  the  training  device  for  this  task.  The  Gemini 
rendezvous  and  docking  simulators  used  to  develop  engineering 
techniques  and  procedures  were  finally  used  for  training. 

Later,  I  will  refer  to  this  concept  of  differential  use  for 
application  to  classroom  learning. 

Recent  Developments  That  Enhance  Utility 

A  number  of  recent  developments  relate  to  the  potential 
use  of  simulators  in  the  classroom.  Some  tend  towards  "gadg- 
etcering,"  but  others  make  the  equipment  more  available  and 
reliable,  and  add  features  that  increase  utility.  I  have  always 
been  leery  of  devices  that  are  overly  complex.  Going  back  to 
my  grade  school  and  high  school  days,  I  rener.ber  that  such 
simple  electronic  equipment  as  the  classroom  radio  and  the  bells 
frequently,  naif unctioned .  I  wonder  if  anything  much  beyond  this 
level  of  complexity  might  not  be  a  maintenance  nightmare  for  the 
classroom? 

Some  recent  developments  are: 

a.  Computer  Availability  -  Digital  computers  are  cheaper 
and  are  tailor-made  to  smaller  applications.  Remote  access  con¬ 
soles  and  time-sharing  capabilities  add  to  their  utility.  Digi¬ 
tal  simulation  has  added  flexibility  because  the  design  is  not 
fixed  initially  and  can  be  reprogrammed  as  new  data  become 
available.  With  analog  devices,  major' design  changes  arc  needed 
to  modify  their  characteristics.  The  hybrid  sinulator  which 
combines  analog  and  digital  characteristics  to  the  bast  advantage 
of  both  has  also  added  to  the  flexibility  of  simulators.  The 
use  of  a  single  central  computer  for  satellite  simulators  or 
the  shared  use  of  a  computer  for  administrative  and  training 
use  has  promise. 
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b.  Flexibility  -  Euilding-block  concepts  have  been  intro¬ 
duced  that  pernit  the  addition  of  features  net  anticipated  or 
else  not  economically  feasible  at  initial  procurement.  This 
allows  one  to  start  snail  and  to  build  to  complex  systems  as 
data  become  available  and  experience  increases. 

c.  Performance  Measurement  -  Scoring  capabilities  have 
improved  because  of  the  stability  of  digital  devices  and  their 
computational  capabilities.  An  example  of  an  increase  in 
measurement  capability  comes  from  one  of  the  simulators  used 
for  research  and  development  at  McDonnell  Douglas.  Initially, 
it  required  six  to  eight  weeks  using  film  scoring  techniques 
to  obtain  performance  data  on  a  five-minute  run.  Refinement 
of  the  device  resulted  in  more  reliable  data  on  performance 
immediately  after  each  run.  Better  measurement  systems  allow 
tailor-made  training  based  on  trainee  proficiency. 

d.  Time  Base  and  Difficulty  Level  -  Special  approaches 
to  modifying  the  time  scale  of  the  training  can  increase  its 
effectiveness.  For  example,  SAMEX,  used  by  the  Army  and  Baylor 
University  to  train  hospital  administrators,  duplicates  for 
training  in  one  to  five  days,  two  years  of  operation  of  a  450 
bed  hospital.  Slow  or  fast-time  techniques  along  with  freezing 
a  situation  and  replaying  the  preceding  portion  have  utility 

in  speeding  the  learning  process.  In  addition,  difficulty  levels 
can  be  changed  through  an  adaptive  process  making  the  problem 
increase  in  difficulty  as  performance  increases.  Adaptive 
systems  are  particularly  useful  for  psychomotor  tasks  and  in¬ 
formation  processing. 

e.  Software  Availability  -  Probably  most  important  for 
classroom  learning  is  not  only  the  simpler  computer  language 
and  programs  now  available  but  the  growing  body  of  software 
that  can  be  adapted  for  simulator  use.  These  come  from  many 
sources  including  data  and  mathematical  models  generated  in 
research  and  development  activities.  The  availability  of  these 
data  can  reduce  the  cost  of  simulating  complex  situations.  Re¬ 
lated  to  this  is  the  fact  that  many  research  and  development 
simulators,  as  mentioned  previously,  are  transitioned  to 
training  use. 

Relation  to  Other  Educational  Techniques 

An  examination  of  the  hierarchy  of  educational  techniques 
suggests  that  simulation  is  one  of  the  least  abstract  and  close 
to  real  life  situations  (4).  Table  1  is  a  gross  classification 
of  educational  techniques  and  materials  to  give  some  indication 
of  where  simulation  fits.  The  degree  of  feedback  to  the  student 
as  a  function  of  his  outputs  generally  increases  as  one  goes 
down  the  list.  For  exanpje,  a  lecture  before  a  large  class 
gives  little  interaction  with  the  instructor,  while  a  tutor  is 
quite  adaptive  and  functions  very  much  as  a  closed-loop  system 
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CLASSIFICATION  OF  EDUCATIONAL  TECHNIQUES  AND  MATERIALS 


PERSONAL 


AUDIOVISUAL  AIDS 


DUPLICATORS 


Lectures 


Written  Materials 


Rote  Responses  Picturcs/Diagraus/Chnrts 


Questions 

Seminars/ 

Discussions 

Tutors 


Audio  Recordings  £  Playback 

Still  Projectors 

Film  Strips 

(Perceptual  Training) 

Motion  Pictures  and 
Television 

ARC  (Tab)  Tests 

Teaching  Machines 


Mock ups /Mode Is /Specimens 
(in  vitro) 

Demonstrations 

Games 

Procedure  Trainers 
Simulators 


Specimens  (in  vivo) 

Job  Samples 

Intcrn/Apprenticeships 


Conputei'-Aidcd  Instruction 
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comparable  to  a  sinulator.  The  last  column  shov:s  duplicators 
of  real  world  situations  beginning;  with  i-ockups  and  models, 
which  are  noureactivc,  and  following  through  to  intern-appren¬ 
ticeships  where  the  man  is  placed  in  the  real  situation  and  gets 
constant  feedback  based  upon  bis  performance. 

The  salient  characteristic  of  simulators  in  the  hierarchy 
is  their  very  high  cost,  not  only  for  development  and  production, 
but  for  operation  and  maintenance.  This  cost  means  that  their 
application  to  the  classroom  situation  will  be  limited  to  areas 
where  their  high  potential  payoff  and  considerable  economic 
benefit  can  be  derived.  Even  though  an  estimated  $GO-billion 
Will  be  spent  on  education  in  1967,  the  average  yearly  expendi¬ 
ture  per  pupil  ist  according  to  the  National  Education  Associa¬ 
tion,  only  $564  in  the  elementary  and  secondary  schools.  Because 
of  the  diversity  and  number  of  governmental  units  and  private 
groups  that  control  these  funds,  there  is  no  single  major  funding 
source  for  the  considerable  cost  to  develop  complex  simulators. 

Present  Applications  in  the  Classroom 

Simulators  similar  to  those  used  for  aerospace  application 
are  seen  rarely  in  the  classroom.  The  moot  court  is  the  best 
example  of  a  high  fidelity  simulation  in  the  classroom.  But 
here  people  effectively  duplicate  the  reactive  elements,  and 
thus  there  is  no  need  for  computers  or  controls  and  displays 
other  than  chairs,  tables,  and  a  bench.  They  are  not  necessary 
to  simulate  the  essential  elements  of  legal  interchange. 

One  explanation  for  the  limited  use  of  simulators  is  that 
it  is  not  the  primary  role  of  the  elementary,  secondary,  or 
university  systems  to  teach  an  individual  to  operate  cr  maintain 
specific  man-machine  systems.  This  training  is  more  appropriate 
for  vocational  schools  and  the  military.  Simulators  for  class¬ 
room  use  will  take  on  a  different  form  than  those  presently  used 
in  the  aerospace  industry. 

Some  Examples  of  Present  Use 

Examples  of  simulators  presently  used  in  the  classroom  are 
given  below.  They  are  all  relatively  simple,  and  their  design 
and  use  have  evolved  gradually. 

a.  Driving  Simulators  -  An  inexpensive  device  used  to 
familiarize  students  with  the  procedures  and  basic  perceptual- 
motor  skills  required  to  operate  an  automobile  (4),  There  are 
problems  related  to  the  utilization  of  driving  simulators.  One 
is  that  the  real  system  is  relatively  cheap  to  produce  even 
when  equipped  with  dual  controls.  In  contrast,  a  high  fidelity 
driving  simulator  would  be  exceedingly  expensive  because  of  the 
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need  to  reproduce  a  reactive  external  visual  scene.  A  high 
fidelity  driving  simulator  has  not  been  Luilt  even  for  research 
in  federal  driving  research  laboratories  because  of  technical 
and  cost  probiens.  Inc  classroom  driving  simulators  are  open- 
loop  devices  that  use  notion  pictures  to  duplicate  the  visual 
scene.  They  do  have  the  capability  for  certain  energencies. 

There  are  704  such  devices  in  classroons  and  they  trained  250,000 
students  in  I960.  In  terns  of  transfer  value,  12  hours  in  the 
simulator  plus  3  hours  in  traffic  is  considered  equivalent  to 
6  hours  in  traffic  (11).  That  is,  12  hours  in  the  simulator 
equals  3  in  traffic. 

b .  Program  Management  Simulation  Exerc i scs  for  Defense 
Weapon  System  Management  (USAP)  -  Exercises  concerned 

with  decision-making  related  to  weapon  system  procurement  have 
training  attractiveness  because  the  time-base  can  be  speeded 
up  tremendously  in  contrast  to  the  life  cycle  of  the  real 
system  (9).  The  future  systems  manager  can  select  alternate 
designs  and  configurations  based  on  cost  and  risk  and  then  re¬ 
late  these  to  outside  criteria  of  effectiveness.  Thus,  he  can 
gain  experience  on  a  number  of  weapon  systems  in  a  short  tine 
period  in  contrast  to  the  real  situation  where  he  would  spend 
5  to  10  years  following  a  single  system.  These  exercises  are 
similar  to  management  games  used  by  the  Anerican  Management 
Association,  IB!!,  and  several  universities  (2,  10). 

c .  Simulated  Man  for  Medical  Training  (DSC  and  Aerojet- 
General)  -  A  manikin  called  '”Sxn  One"  is  used  for 

training  residents  in  anesthesiology  (3).  Critical  physiological 
parameters  are  displayed  on  a  read-out  panel.  Cardiovascular 
sounds  are  present,  and  the  manikin  responds  to  drugs  and  anes¬ 
thetics  in  critical  areas  such  as  pupil  size.  Hybrid  computers 
are  employed.  Emergency  situations  such  as  heart  arrest  and 
fibrillation  are  possible.  The  mouth,  throat,  and  other  essen¬ 
tial  anatomical  features  are  reproduced  precisely.  The  device 
was  developed  under  a  $272,000  grant  from  the  Office  of  Educa¬ 
tion,  Some  biomedical  specialists  feel  that  enough  is  known 
about  some  other  body  systems  to  simulate  them  in  an  equivalent 
manner. 

d .  Equivalent  Response  Learning  (Hew  York  Institute  cf 
Technology )-  Relatively  simple  devices  train  keyboard 

operators  using  a  cuing  technique  that,  activates  the  keyboard 
to  instill  proper  manual  responses  (9), 

Considerations  in  the  Transition  from 
Aerospace  to  Classroom  An p.lications 

The  first  step  in  identifying  where  present  aerospace 
Technology  might  be  adapted  to  the  classroom  should  be  based  on 
a  systems  approach  which  is  an  organized  effort  to  examine  the 
classroom  situation,  identify  relevant  variables,  and  the 
desired  output.  These  are  then  related  to  present  and  future 
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technology,  end  the  potential  system  is  assessed  finally  on 
a  eost/beneflt  basis.  Such  an  approach  is  beyond  the  scope 
of  this  paper,  but  it  probably  would  not  identify  many  more 
areas  for  new  application  than  those  presently  applied  in  the 
elassrooa  because  ofs 

a.  Inappropriateness  since  such  of  what  is  taught  in  the 
elassrooa  does  not  lend  itself  to  siaulation. 

.  b.  Very  high  cost  which  cannot  be  borne  by  the  already 
overburdened  educational  systea  unless  striking  benefits  are 
possible. 

Vhat  are  soae  of  these  benefits  that  oust  be  realized? 

A  siaulator  oust  be  unique  to  justify  its  cost.  Aircraft 
slaulators  were'  justified  initially  because  they  allowed 
realistic  practice  on  eaergencies  too  hazardous  to  reproduce 
in  the  air.  Thus  they  had  a  unique  capability  that  could  save 
aoney  in  terns  of  accident  reduction.  Soae  factors  that  can 
be  used  to  justify  slaulators  aret 

a.  Consequence  of  Inadequate  Performance  -  Inproper  per- 
foraance  is  reflected  either  in  econonlc  loss  or  fatalities/ 
injuries,  such  as  that  by  a  coaaercial  pilot. 

b.  Cost  of  Using  Actual  System  for  Training  -  The  cost 
of  operating  the  systea  such  as  an  aircraft  is  high  both  in 
terns  of  revenue  loss  and  salaries. 

c.  Hazard  of  Using  Actual  System  for  Training  -  The  system 
in  training  use  has  an  undesirable  effect  on  society  such  as  a 
nuclear  power  plant  failure,  a  crashed  commercial  aircraft 
damaging  civilian  areas,  or  the  annoyance  of  sonic  booms. 

d*  Availability  of  Actual  System  for  Training  -  The  equip¬ 
ment  is  not  available  for  training  such  as  a  spacecraft  which 
Bust  be  flown  correctly  the  first  time  or  the  transition  of  a 
pilot  to  a  single-place  aircraft  where  the  first  flight  is 
without  direct  supervision. 

e.  Capability  of  Actual  System  to  Provide  Emergency 
training  -  0ut-o£-tolerance  conditions  cannot  be  re¬ 
produced  safely  without  damaging  the  real  system  or  can  be  re¬ 
produced  only  at  great  expense,  but  can  be  reproduced  readily 
in  a  simulator. 

f.  Requirement  to  Integrate  Complex  Team  Activity  -  The 
integration  of  the  activities  of  a  large  number  of  people  in 
terms  of  information  processing  and  decision-making  is  required 
particularly  when  complex  information  must  be  assembled  and 
reacted  on  quickly. 
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g.  Susceptibility  of  Task  to  Stress  -  Tasks  which  oust 
be  perforued  under  stressful  environmental  conditions  such  as 
those  involved  in  a  space  flight  must  be  learned  to  a  high 
level  of  proficiency  to  avoid  disruption  and  this  level  is 
usually  difficult  to  obtain  on  the  real  systen  because  of  the 
iapracticality  of  repeating  critical  segments  for  training. 

h.  Capability  to  Centralize  Training  Location  -  The  capa- 
bility  to  reduce  the  number  of  training  locations  from  many  to 

a  few  with  a  simulator  can  reduce  cost  and  achieve  standardization. 

i.  Face  Validity  of  Training  -  If  trainee  acceptance  is 
necessary  to  assure  their  participation  as  with  pilots  or  func¬ 
tional  illiterates,  then  the  "realism"  of  a  simulator  can  be 
important. 

j.  National  Meed  -  The  need  for  specialized  skills  to 
support  a  space  program  or  medical  technicians  for  developing 
nations  can  be  important  enough  in  terms  of  well-defined  national 
goals  that  federal  funding  for  simulation  equipment  might  be 
required  to  speed  up  the  training  process. 

Survey  of  Potential  Areas 

A  variety  of  classification  schemes  might  be  used  to  examine 
the  potential  for  simulators  in  the  classroom.  One  might  use 
perceptual-motor,  procedural,  decision-making,  and  personnel 
interaction  skills  for  primary,  secondary,  university,  graduate, 
professional,  and  vocational  training. 

With  such  a  scheme,  many  areas  for  application  can  be  dis¬ 
missed  quickly  as  inappropriate  based  upon  a  screening  using 
the  criteria  of  applicability.  A  summary  of  such  a  gross 
screening  follows: 

a.  Basic  Skills  -  There  would  seem  to  be  little  applica¬ 
bility  to  the  initial  acquisition  of  skills  (6).  Books,  films, 
tapes,  and  programmed  instruction  are  more  than  adequate  as 
teaching  aids. 

b.  Remedial  for  Basic  Skills  -  Simulation-like  devices 
might  be  used  for  perceptual  training,  but  the  applicability 

of  true  simulation  devices  would  be  limited  unless  face  validity 
is  necessary  to  motivate  an  adult  population  such  as  functional 
illiterates . 

c.  Handicapped  Training  -  Handicapped  persons,  particularly 
those  with  performance  problems,  might  be  trained  using  simula¬ 
tion  devices  that  duplicate  some  of  the  real  tasks  they  must 
perform.  An  example  might  be  driving  a  car  where  repetition 
training  is  needed  on  some  task  elements  to  overcome  handicaps. 
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d.  Vocational  -  The  military  has  paved  the  way  for  the 
use  of  relatively  low  cost  simulators *  but  there  is  a  question 
of  whether  these  are  cost/effective  for  civilian  training  situa¬ 
tions*  Some  areas  that  might  be  considered  are  TV  repair* key¬ 
board  operators,  and  domestic  and  foreign  medical  technicians. 

The  latter  could  meet  the  criteria  of  national  need  because  of 

a  political  commitment.  Other  possible  areas  could  involve 
the  telephone  information  operator  who,  in  the  near  future,  will 
probably  need  computer  assistance  to  search  directories.  Bell 
Telephone  Laboratories  has  constructed  an  engineering  simulation 
to  investigate  the  feasibility  of  computer-assisted  search  which 
could  be  adapted  for  training  use  (1). 

e.  Professional  -  Law  uses  the  moot  court,  and  "Sim  One" 
has  been  described  as  a  specific  technique  for  medical  training. 
However,  the  area  of  greatest  potential  probably  relates  to 
training  in  medical  diagnosis. 

f.  Operation  of  Specific  Equipment  -  Railroad  locomotives, 
nuclear  power  plants,  power  distribution  systems*  and  chemical 
plants  are  all  critical  in  terms  of  both  safety  and  the  economics 
of  operation.  Errors  can  have  significant  consequences.  In  the 
case  of  railroads,  firemen  are  not  available  for  the  apprentice 
route  formerly  used  to  produce  engineers.  Simulators  might  be 
used  to  train  a  new  engineer  in  the  operation  of  the  locomotive 
and  in  road  familiarization. 

g.  Management  Decision-Making  -  Applications  might  involve 
low  to  high-level  economic  decisions  ranging  from  inventory  con¬ 
trol  to  the  impact  of  a  change  in  interest  rates  on  the  tax  base. 
Mar  gaming  now  uses  computers  to  simulate  the  complexities  of 

a  military  decision,  either  as  a  part  of  a  complex  command  and 
control  system  as  SAGE,  or  without  reference  to  a  specific  com¬ 
mand  system.  One  advantage  of  management  system  simulations  is 
that  they  can  demonstrate  cause-effect  relationships  to  the 
student  that  might  be  masked  in  the  real  system  because  of  its 
complexity.  Further  increases  in  the  complexity  of  these  simu¬ 
lations  can  be  expected. 

Future  Possibilities  for  the  Classroom 

It  is  apparent  that  there  are  no  obvious  and  exciting  new 
areas  for  future  application  to  the  classroom.  However,  there 
is  a  common  element  that  seems  to  permeate  present  and  future 
use  that  bears  examination.  This  relates  to  the  development 
of  integrative  skills*  and  the  reasoning  goes  something  like 
this:  individual  skills  and  content  such  as  mathematics* 

anatomy*  and  .thermodynamics  are  taught  on  a  part  or  isolated 
basis  by  conventional  techniques.  The  application  of  these 
skills  and  knowledge  is  another  matter.  This  is  usually  accomp¬ 
lished,  sometimes  haphazardly,  and  sometimes  through  apprentice¬ 
ships  or  internships. 
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As  an  example,  an  engineer,  despite  the  skills  learned 
from  university  training,  must  spend  many  years  developing 
the  experience  necessary  to  apply  selectively  to  the  design 
of  the  system  information  that  he  has.  This  process  gained 
by  experience  allows  him  to  make  trade-offs  within  the  tech¬ 
nical  constraints  and  relate  these  to  schedule,  cost,  and 
performance.  This  skill  in  system  integration  seems  to  be 
based  on  experience  rather  than  on  what  is  taught  in  the  class¬ 
room.  Industry  tends  to  measure  an  engineer's  potential  and 
salary,  not  on  courses  taken  or  his  grades  in  the  university, 
but  rather  on  specific  experience. 

It  is  conceivable  that  another  level  of  training  can  be 
achieved  with  classroom  simulators  which  allow  the  student 
to  develop  the  "experience"  needed  to  solve  actual  problems. 

That  is,  to  allow  him  to  develop, while  still  in  school, approaches 
derived  from  his  testing  alternatives  in  complex  situations 
that  not  only  integrate  existing  knowledge  but  allow  him  to 
experience  the  impact  of  schedule  changes,  failures,  funding 
changes,  and  new  requirements  on  the  application  of  knowledge. 
Thus  the  simulation  would  allow  experience  which  better  approxi¬ 
mates  the  conditions  under  which  knowledge  must  be  applied  later. 
A  similar  kind  of  problem  is  seen  with  the  new  Ph.D.  who,  when 
asked  to  apply  his  research  skills  in  an  applied  situation,  is 
not  flexible  and  experienced  enough  to  identify  critical  issues 
and  variables.  His  approach  tends  to  be  formula  oriented  and 
rigid  rather  than  adaptive  to  the  situation. 

Gaining  experience  is  more  than  reading  or  watching  someone 
else  do  the  job.  It  seems  to  require  direct  involvement  in 
applying  the  information  where  feedback  occurs  regarding  the 
consequence  of  the  application.  Simulators  are  ideal  for  this 
function  since  they  allow  the  student  to  participate  directly 
in  near-real  activities.  For  example,  a  number  of  aviation 
psychologists  who  are  not  pilots  learned  to  fly  complex  aircraft 
simulators.  Even  though  they  had  not  flown  a  high  performance 
aircraft,  this  experience  allowed  them  to  obtain  a  detailed 
understanding  of  the  system  and  its  characteristics  way  beyond 
that  obtained  from  doing  a  task  analysis,  reading  about  flying, 
or  watching  someone  else  fly. 

The  Emerging  Data  and  Technology 

The  major  drawback  to  the  applications  just  discussed  re¬ 
lates  to  the  cost  particularly  for  software  and  system  models. 

The  potential  for  the  classrooms  becomes  feasible  if  other 
funding  sources  develop  the  appropriate  data  and  system  models. 
Classroom  simulation  of  complex  functions  becomes  feasible  as 
software  and  computer  models  are  developed  by  others  in  a  form 
that  can  be  adapted  without  prohibitive  expense.  For  example, 
in  the  space  program  automated  laboratories  and  experiments 
are  being  developed,  and  physical,  social,  financial,  biologi- 
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calt  demographic,  and  personnel  data  banks  are  becoming  more 
and  more  available.  The  point  is  that  the  data  and  computer 
models  now  used  in  many  research  programs  can  be  adapted  to 
classroom  use  allowing  the  student  to  gain  fresh,  firsthand 
knowledge  of  complex  problems. 

Some  of  the  data  and  computer  models  in  the  past  have 
been  relatively  simple  and  contained  too  few  variables  to  be 
sensitive  to  critical  interactions.  But  more  complex  models 
are  now  in  use.  for  example,  McDonnell  Douglas  has  developed 
from  empirical  data  a  computer  model  based  on  SIMSCRXPT,  a 
simulation  programming  language,  of  an  emergency  room  in  a 
specific  hospital  that  allows  assessment  of  the  impact  of 
variables  such  as  mass  emergencies,  new  facilities,  and  staffing 
changes  (8).  This  kind  of  model  could  be  used  in  the  classroom 
to  train  hospital  administrators  to  understand  the  dynamics  of 
emergency  room  operation.  An  important  aspect  of  any  simulation 
for  classroom  use  is  the  availability  of  measurement  systems 
that  allow  feedback  of  performance  for  knowledge  of  results  and 
grading  purposes. 


Some  Examples  of  future  Use 

If  research  and  development  models  and  supporting  computer 
data  are  to  be  adapted  to  classroom  use,  then  techniques  are 
needed  to  allow  the  student  access  to  the  information.  The 
following  shows  the  gross  characteristics  of  such  a  system: 


MATHEMATICAL 
SYSTEMS  MODEL 


ITZZETZT 


PROflCIEHCY 

ASSESSMENT 

1 

COMPUTER 

■ 

DATA  BANK 

Knowledge  of  results 

H 

Digital  or  Hybrid 

' Model  Related 
— ’►Subject  Matter  Related 

Grading 

i 


DISPLAYS 

CONTROLS 

Alphanumerical 

Data  Search  and  Display 

Pictorial 

Model  Activation 

Physical  Model 

Time  Base  Variation 

Parameter  Variation 

Display  of  Results 

, 

-  -  ~T  ~ 

Assessment  of  Results 

A 

TRAINEE ( S ) 


Essentially,  the  student  would  have  a  console  that  allows 
him  to  enter  the  system  model,  manipulate  it,  and  obtain  feedback 
of  his  actions  either  on  a  heuristic  or  preplanned  basis.  The 
student  could  test  limits,  introduce  forcing  functions,  or 
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determine  sensitivities  by  introducing  various  conditions. 

Through  this  process,  he  could  rapidly  examine  the  interactions 
of  complex  information  and  determine  the  critical  variables 
within  a  system.  This  process  is  analogous  to  the  one  used 
frequently  in  systems  engineering.  Several  examples  for  appli¬ 
cation  are: 

a.  Research  -  Training  in  research  can  be  a  wasteful 
process.  An  ind5 vidual  spends  years,  sometimes,  before  any 
feedback  is  obtained  regarding  the  efficacy  of  his  approach. 
Training  should  make  him  more  sensitive  early  to  research  issues 
by  allowing  the  simulation  of  a  variety  of  problems.  He  could 
apply  multiple  approaches  to  data  to  test  insights  quickly  and 
then  assess  all  of  these  against  "knowns"  similar  to  those  used 
by  the  chemist  in  teaching  quantitative  analysis.  Such  a  process 
would  train  him  in  the  strategies  of  research  and  allow  him  to 
quickly  isolate  critical  variables.  Practice  using  "automatic 
libraries"  could  complement  such  training. 

b.  Medicine  -  Sophisticated  mathematical  models  are  begin¬ 
ning  to  exist  for  systems  of  the  body  which  give  observable 
responses  in  physiological  states  when  model  parameters  are  varied. 
The  fact  that  "Sim  One"  could  be  developed  indicates  such  a  capa¬ 
bility.  Students  learning  the  systems  of  the  body  could  gain 
experience  through  a  simulation  in  the  interactive  effects  that 
occur  through  disease  when  drugs  are  introduced.  The  pulmonary, 
cardiovascular,  and  renal  systems  are  potentials  for  simulation. 

A  more  obvious  area  relates  to  the  use  of  simulators  to  teach 
diagnosis . 

c.  Engineering  -  Engineering  training  tends  to  stress  the 
acquisition  of  isolated  skills  and  knowledge.  Training  in  ap¬ 
plication  occurs  after  the  student  leaves  the  university.  Many 
tines  he  is  ill-equipped  for  considering  the  complex  interactions 
that  may  occur  within  the  elements  of  his  technology  as  well  as 
their  impact  on  the  social  and  economic  structure  of  society. 
Computer  models  could  allow  these  interactions  to  be  defined 
better.  For  example,  one  could  assess  the  impact  of  a  new 
highway  not  only  on  traffic  flow  but  on  the  area’s  economic  and 
social  structure. 


CONCLUSIONS 


Present  and  projected  capabilities  in  aerospace  simulation 
were  examined  to  determine  what  aspects  might  be  adapted  for 
classroom  use.  The  attributes  of  simulators  as  well  as  the 
criteria  that  must  be  met  before  they  can  be  adopted  were  considered. 

Simulators  are  expensive  to  acquire  and  operate.  Their 
justification  must  be  unique.  It  appears  that  beyond  a  few  con¬ 
ventional  uses  such  as  for  driver  training  that  they  could  be 
applied  best  to  the  integration  of  complex  skills  and  knowledge. 

This  process  would  be  relatively  economical  if  existing  computer 
models  from  research  and  development  programs  are  adapted  to 
classroom  use. 
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THE  HEW  LOOK  IK  EDUCATIOKAL  TECHNOLOGY 


IN  TECHNICAL  TRAINING 
G.  Douglas  Kayo 


The  trend,  or  new  look  in  educational  technology  in 
technical  training,  seems  to  me  to  be  unusually  clear,  con¬ 
sidering  the  early  stage  of  its  development.  This  trend  is 
away  from  mass,  lock-step  training  and  toward  individually 
adapted  training,  carefully  geared  to  performance  requirements. 
When  we  move  from  this  general  statement  to  specific  details, 
the  projection  into  the  future  becomes  less  clear.  But  even 
so,  there  is  a  principle  which  tends  to  determine  the  direc¬ 
tion  of  movement.  Given  an  acceptable  quality  of  training, 
the  economics  of  the  situation  constantly  maneuvers  technical 
training  toward  the  most  efficient  means  of  achieving  the 
aims  of  the  training  involved.  This  stems  to  a  large  degree 
from  the  fact  that  the  recipient  of  technical  training, 
whether  in  the  military  or  in  industry,  is  paid  to  take  the 
training.  He  is  there  because  management  wants  him  to  perform 
tasks  that  he  could  not  perform  prior  to  training.  Understand¬ 
ably,  management  wants  him  to  learn  the  desired  skills  and 
to  begin  performing  them  as  soon  as  possible. 

Other  factors  will  influence  the  direction  of  technical 
training  to  a  greater  or  lesser  extent  from  time  to  tine,  but 
the  primary  factor  over  the  longer  term  is  primarily  economic. 

In  using  this  term  I  do  not  imply  a  "penny  wise"  point  of  view. 

I  use  the  term  to  connote  simply,  "the  effective  allocation 
of  limited  resources."  Thus,  the  commitment  of  substantial 
funds  to  develop  and  implement  an  educational  technology 
ordinarily  is  not  a  problem  if  it  appears  that  the  goals 
of  training  can  be  achieved  more  efficiently. 

In  attempting  to  project  the  future  of  educational  tech¬ 
nology  in  technical  training  my  views  will  be  guided  by  the 
economic  principle  stated  above,  or  if  you  prefer,  assumption. 

In  addition  I  shall  limit  my  projection  into  the  future  to  a 
typical  military  technical  training  situation,  although  I 
suspect  that  most  technical  training  will  be  affected  by 
essentially  the  same  forces. 

Individually  adapted  training,  which  doubtless  will  be 
machine  facilitated,  will  be  adaptive  in  two  respects:  first, 
in  terms  of  trainee  aptitude,  interests,  and  other  pertinent 
characterii tics ;  and  secondly,  in  terms  of  the  specific  ter¬ 
minal  performance  that  is  desired' - by  management.  In  the 
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first  instance  the  instructional  system  is  likely  to  be 
siailar  to  computer  assisted  instruction  in  non-technical 
training  situations.  I  expect  it  to  reach  a  nature  state 
through  a  series  of  steps  which  will  include  initial  use  of 
the  conputer  largely  to  acconplish  the  routing  of  the  in¬ 
dividual  trainee  through  training,  beginning  with  his  initial 
level  of  knowledge  and  skill  and  proceeding  to  the  desired 
level  in  as  nearly  an  optinal  nanner  as  can  be  accomplished. 

This  optinal  routing  will  require  a  great  deal  of 
developnent  effort  in  each  instructional  system.  Initially 
much  of  the  instructional  materials,  such  as  programmed 
booklets,  audio  recordings,  films,  and  video  tapes,  are 
likely  to  be  structurally  independent  of  the  computer.  This 
is  due  to  technical  and  economic  reasons.  Gradually  the  in¬ 
structional  materials  will  be  structurally  integrated  into 
the  computer  based  system. 

In  the  second  instance,  which  involves  tailoring  training 
to  the  performance  requirements  of  the  specific  job  which  the 
trainee  will  fill,  a  number  of  changes  from  our  present  pro¬ 
cedures  are  foreseen.  For  example,  during  the  recruit  train¬ 
ing  period  the  specific  position  in  a  given  squadron  to  which 
the  man  will  be  assigned  will  be  determined.  The  specific 
performance  requirements  of  the  position  will  be  determined 
and  the  programming  of  the  man's  training  to  meet  these 
requirements  will  begin.  The  trainee  will  proceed  as  rapidly 
as  his  aptitude  and  motivation  will  permit  through  the  required 
training,  to  the  end  of  his  individual  course,  and  to  the 
position  for  which  he  was  selected. 

Suppose  the  trainee  is  not  intrinsically  motivated. 

Suppose  he  is  content  to  make  a  "career  of  training"  in  a 
course  that  normally  should  be  completed  in  a  few  weeks.  There 
will  be  time  standards  at  three  levels:  minimum,  standard,  and 
superior.  Present  corrective  procedures,  such  as  conferences, 
evening  sessions,  and  termination  of  training,  will  continue 
in  the  case  of  failure  to  meet  the  minimum  level  of  performance. 
A  positive  incentive  to  achieve  beyond  the  standard  time  re¬ 
quirement  will  be  included  in  the  form  of  a  monetary  reward 
for  rapid  achievement  of  the  objectives  of  the  course.  This 
reward  will  be  based  upon  a  percentage  of  the  amount  saved  by 
early  completion  of  the  course.  Trainees  will  be  permitted 
to  work  longer  hours  than  those  of  the  standard  work  day  if 
they  wish  to  do  so  and  will  be  rewarded  for  early  completion 
of  the  course. 


The  units  of  which  the  course  for  each  trainee  will  be 
constructed  will  be  detei'nined  by  "specific  behavioral  objec¬ 
tive"  or  "terminal  performance  specifications,"  as  is  cur¬ 
rently  true  in  programmed  instruction.  These  objectives  will 
be  assembled  for  each  individual  in  response  to  the  question 
of  what  tasks  oust  be  performed  in  the  position  to  which  the 
nan  will  be  assigned. 

There  will  be  no  instructors  as  we  now  know  then.  There 
will  be  three  primary  instructional  tasks.  The  first  will 
be  preparation  (including  validation)  of  adaptive  material 
to  meet  specific  behavioral  objectives.  These  materials  will 
include  extensive  branching  and  remedial  loops.  The  second 
instructional  task  will  involve  selecting  the  behavioral  ob¬ 
jectives  which  pertain  to  the  position  for  which  the  man  was 
selected.  It  is  at  this  point  that  the  content  of  the  course 
is  tailored  to  the  requirements  of  the  position.  Huch  of  the 
information  required  to  make  a  decision  concerning  what  the 
content  of  each  course  should  be,  will  be  stored  on  discs  and 
available  from  the  computer.  The  third  instructional  task 
will  bo  the  management  and  monitoring  of  the  training  situa¬ 
tion.  The  monitoring  function  will  be  accomplished  in  much 
the  same  way  that  automated  production  machines  are  monitored 
at  the  present  time,  although  a  "human  touch"  will  be  included 
in  the  management  function. 

The  extent  to  which  it  will  be  economically  feasible  to 
decentralize  training  is  not  entirely  clear.  Progress  made 
in  economical  communication  from  the  computer  to  remote  loca¬ 
tions  will  be  a  factor.  However,  a  larger  factor  in  technical 
training  will  be  the  duplication  of  costly  training  equipment. 
Until  relatively  inexpensive,  but  effective,  substitutes  for 
such  equipment  are  developed,  technical  training  doubtless 
will  continue  to  be  centralized  in  training  centers. 

I  have  attempted  briefly  to  point  to  some  of  the  changes 
that  may  accompany  the  development  of  new  educational  technol¬ 
ogy  in  technical  training.  Obviously  most  of  the  detailed 
projections  are  largely  speculation.  One  point  that  I  feel 
is  clear,  however,  is  that  the  rate  of  change  which  has  been 
so  slow  in  the  past  as  to  be  almost  imperceptible,  will  be 
greatly  accelerated  in  the  future. 


LEARNING  OPERATIONAL  EQUIPMENT  AS 
A  CRITERION  IN  TRAINING  RESEARCH 

G.  Douglas  Hayo  and  Alexander  A.  Longo 


It  has  been  said  that  training  and  education  differ, 
nftt  so  much  in  terms  of  methods  and  techniques  as  in  the 
specificity  of  their  objectives.  This  statement,  we  e  eve, 
cln  be  extended  to  research  pertaining  to  education  and  t 
training,  as  well.  While  personnel  engaged  in  training  r 
search  would  prefer  that  their  findings  be  generalixable 

over  a  wide  range  of  educational  and  tra*?x*g  ***?***°nS,The 
they  do  not  consider  this  to  be  an  ei ssential  condition.  Th 
study  reported  here  is  a  case  in  point,  and  in  fact  capi- 
talizes  upon  a  condition  that  is  essentially  peculiar  to 
iaial  aviation  technical  training.  It  does  have  some  fea- 
J*.  however,  that  are  interesting  to  us,  and  perhaps  will 
others  Primary  among  these  features  are  the  implemen- 
Jition  o?  ?hi  concept  ability  to  learn  operational  equip¬ 
ment  as  a  criterion,  and  the  construction  of  ®  ^ions°in  a 
measure  based  upon  terminal  performance  specifications  in 
represen tat i v e  Operational  equipment  course  These  two 
topics  will  be  described  in  greater  detail  in  a  subsequen 

part  of  the  paper. 

This  is  the  third  in  a  series  of  four  reports  describing 
*  «tulv  which  compared  the  performance  of  students  receiving 
of  different  lengths  for  the  Navy  rating  or  occupa- 
of  Aviation  Electronics  Technician  R  (Radar  and  Radar 

^^f^CH^e  ^efn°at  -"-“  raining 

and  occupational  "pipeline.-  The  first  »eas»re  was 

nisTMi  :s:ocnnL:rrr:as"tah:r«"thie:nd  .* .  5xir-« 

course  in  which  radar  equipment  was  used  as  a  neans  of  teaching 
principles  and  theory  of  radar. 

The  present  paper  is  based  ^  "uipnL" 

in^operating  squadrons  i-^y^^iUp-v^^f-a- 

tio^concernin^the^p.rfornance^i^operating  squadrons  of 

personnel  receiving  different  amounts  of  training. 


-47- 


One  group  received  the  original  aviation  electronics 
fundamentals  course,  which  was  19  weeks  in  length,  and  then 
received  the  original  radar  course,  11  weeks  in  length,  for 
a  total  of  30  weeks  of  training  before  being  assigned  to  a 
squadron  and  receiving  a  brief  course  in  the  specific  equip¬ 
ment  used  by  the  squadron.  The  oLher  group  received  a  revised 
and  shortened  course,  both  in  aviation  electronics  fundamentals 
and  in  radar  training.  The  revised  fundamentals  course  was 
14  weeks  in  length  and  the  revised  radar  course,  8  weeks  in 
length,  for  a  total  of  22  weeks.  Thus  the  students  assigned 
to  the  revised  courses  received  8  weeks  less  training  than 
did  the  students  assigned  to  the  original  courses*. 

It  is  desirable  for  personnel  to  spend  as  small  a  pro¬ 
portion  of  their  time  in  training  status  as  possible,  con¬ 
sistent  with  their  acquiring  the  knowledge  and  skill  necessary 
to  performing  adequately  on  the  job.  In  general,  time  spent 
in  training,  not  only  is  not  immediately  productive  but  requires 
diversion  of  material  resources  and  support  personnel  from 
productive  tasks.  This  matter  has  been  accentuated  in  recent 
years  by  increased  requirements  for  training,  stemming  from 
more  complex  equipment  on  the  one  hand  and  low  reenlistment 
rate  of  first  enlistment  personnel  on  the  other.  Thus,  it  is 
quite  important  that  personnel  be  adequately  trained  and  at 
the  same  time  that  training  of  first  enlistment  personnel  be 
held  to  a  minimum,  consistent  with  satisfactory  performance. 

The  basic  training  plan  which  has  emerged  in  naval  avia¬ 
tion  technical  training  retains  the  economy  inherent  in  mass 
production  for  general,  theoretical  training,  but  recognizes 
that  formalized  training  must  be  given  on  the  specific  equip¬ 
ments  and  systems  the  man  will  be  expected  to  maintain  in 
order  for  him  to  perform  satisfactorily.  Thus,  following 
general,  theoretical  training  in  the  basic  schools,  personnel 
are  assigned  to  operating  squadrons  but,  as  a  rule,  are 
trained  on  the  equipments  used  by  these  squadrons  before  re¬ 
porting  to  the  squadron.  This  training  is  conducted  at  the 
naval  air  station  at  which  the  operating  squadron  is  stationed 
when  ashore  by  Combat  Readiness  Air  Groups,  which  in  turn 
delegate  most  of  the  responsibility  for  formalized  maintenance 
training  of  enlisted  personnel  to  the  Naval  Air  Maintenance 
Training  Detachment  located  at  the  same  naval  air  station. 

The  Combat  Readiness  Air  Group  has  the  same  type  of 
operational  aiicraft  as  the  squadrons  for  which  they  train, 
but  most  of  the  training  of  maintenance  personnel  is  conducted 
on  training  panels  which  are  developed  by  the  manufacturer  of 
the  aircraft,  complete  with  all  training  materials  required. 
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These  training  panels  are  essentially  the  sane  as  the  corres¬ 
ponding  equipment  or  system  in  the  aircraft  but  display  the 
equipment  in  such  a  way  that  an  understanding  of  the  system 
and  maintenance  procedures  pertaining  to  it  are  more  readily 
acquired  than  would  be  the  case  if  the  actual  aircraft  were 
used* 

Since  the  primary  function  of  general,  theoretical  train¬ 
ing  in  the  basic  or  Class  A  schools  is  to  train  to  the  point 
that  the  graduate  of  the  school  can  readily  assimilate  train¬ 
ing  on  the  operational  equipment  and  systems  he  will  be  ex¬ 
pected  to  maintain,  a  measure  of  ability  to  acquire  information 
and  skills  pertaining  to  such  operational  equipment  becomes  a 
logical  criterion  in  the  evaluation  of  basic  school  training. 

When  viewed  in  this  manner,  the  criterion  of  performance 
in  operational  equipment  courses  has  certain  advantages,  as 
follows : 

(1)  A  substantial  period  of  behavior  observation  is 
provided,  longer  than  ordinarily  would  be  possible  in  a  per¬ 
formance  testing  situation* 

(2)  The  behavior  observed  is  directly  related  to  the  ob¬ 
jectives  of  the  general,  theoretical  training  and  to  the  v/ork 
the  man  will  be  doing  on  the  job. 

(3)  The  observation  of  behavior  occurs  before  further 
training  and  experience  have  an  opportunity  to  exert  a  differ¬ 
ential  effect  upon  the  various  individuals  in  the  group. 

If  performance  in  operational  equipment  courses  is  to 
serve  as  an  adequate  measure  of  the  effectiveness  of  training 
previously  received  in  the  basic  schools,  certain  conditions 
must  be  met.  First,  it  is  necessary  to  select  an  equipment 
course  that  is  representative,  in  terms  of  content  and  diffi¬ 
culty,  of  the  equipment  courses  to  which  graduates  of  the 
basic  school  are  assigned.  This  is  true  because  it  is  diffi¬ 
cult  to  compare  graduates  attending  a  number  of  different 
courses.  Following  the  selection  of  a  representative  equipment 
course,  the  input  to  the  course  must  be  controlled  to  ensure 
that  the  personnel  assigned  to  the  course  are  representatives 
of  the  treatment  groups  to  which  we  wish  to  generalize.  Thus 
members  of  the  groups  which  we  wish  to  compare  may  be  acsigne 
to  the  representative  equipment  course  on  a  random  basis  or 
they  may  be  matched  on  the  basis  of  a  pertinent  measure  taken 
before  the  beginning  of  the  course.  Finally,  it  will  usually 
be  necessary  to  develop  or  revise  the  measures  used  to  assess 
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performance  in  the  equipment  course  in  order  to  ensure-  their 
adequacy  as  a  criterion  measure.  In  those  instances  in  which 
the  conditions  just  described  can  be  ret,  ability  to  learn 
oper*ational  equipment  and  systems  lu'.s  considerable  appeal  as 
a  criterion  in  training  research. 

The  sample  consisted  of  58  students  i:ho  graduated  from 
the  Aviation  Electronics  Technician  R  (Radar)  Course  at  the 
Naval  Air  Technical  Training  Center,  Memphis ,  Tennessee, 
between  August  and  November  1SC5.  All  subjects  were  regular 
Navy  personnel  who  had  had  no  previous  Navy  experience.  The 
mean  civilian  educational  level  of  the  group  was  approximately 
12  years  and  means  on  pertinent  aptitude  tests,  such  as  the 
Navy  General  Classification  Test,  Arithmetic  Test,  and  Elec¬ 
tronics  Technician  Selection  Test,  were  in  a  60-65  range. 

These  scores  are  Navy  standard  scores,  which  in  the  Navy 
standardization  group  had  a  mean  cf  50  and  a  standard  devia¬ 
tion  of  10. 

A  matched  group  design  was  employed  which  utilized  2S 
pairs  of  subjects.  The  matching  variable  was  the  grade  mace 
in  the  Aviation  Fundamentals  School,  a  course  two  weeks  in 
length,  completed  immediately  prior  to  training  in  electronics 
fundamentals.  The  correlation  between  the  matching  variable 
and  the  primary  measure  of  performance  in  the  present  study 
was  .41  in  the  case  of  the  group  that  received  the  original, 
longer  course  in  electronics  fundamentals  and  radar,  and  .24 
in  the  case  of  the  group  that  received  the  revised,  shorter 
course . 

The  original  design  called  for  30  pairs  of  subjects,  but 
after  the  subjects  had  been  assigned,  one  man  became  ill  and 
it  was  necessary  to  eliminate  this  pair  of  subjects  from  the 
study.  The  two  treatment  groups  were  enrolled  in  the  course 
at  different  times.  The  first  group  previously  had  received 
thirty  weeks  of  training  in  electronics  fundamentals  and 
principles  of  radar  equipment,  as  described  in  a  previous 
section  of  this  paper.  Six  students  from  each  of  five  classes 
completing  the  Aviation  Electronics  Technician  R  (Radar)  Course 
during  August  and  September  1965  were  assigned  to  a  one  week 
course  in  the  coder  group  portion  of  the  P-3A  IFF/boran  System, 
APA-89.  This  course,  which  consisted  of  36  hours  of  actual 
instruction,  was  conducted  by  the  Naval  Air  Maintenance 
Training  Department  at  the  Naval  Air  Station,  Patuxent  River, 
Maryland. 

The  purpose  of  the  course  v:as  to  provide  maintenance 
personnel  with  instruction  on  the  latest  maintenance  techniques, 
modifications,  operational  systems,  and  circuit  analysis  of  the 
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coder  group  portion  of  the  IFF/Loran  System  in  the  P-3A  air¬ 
craft.  Upon  completion,  maintenance  personnel  were  expected 
to  be  able  to  diagnose,  troubleshoot,  maintain,  and  service 
the  coder  group  portion  of  the  IFF/Loran  System  and  know 
the  safety  precautions  to  be  observed. 

Because  of  the  nature  and  size  of  the  equipment  involved, 
the  maximum  number  of  students  that  could  be  enrolled  in  the 
course  at  a  given  time,  was  six.  Selection  for  the  course 
was  made  about  four  weeks  prior  to  graduation  from  the  Avia¬ 
tion  Electronics  Technician  R  (Radar)  Course.  The  subjects 
were  selected  in  a  manner  which  would  provide  a  representative 
range  of  scores  on  the  matching  variable.  The  members  of  the 
other  treatment  group,  the  group  that  received  22  weeks  of 
electronics  fundamentals  and  radar  principles  training,  com¬ 
pleted  the  Aviation  Electronics  Technician  R  (Radar)  Course 
during  October  and  November  1965.  Six  graduates  of  the  course 
from  each  of  five  classes  during  this  period  were  assigned  to 
the  specific  equipment  course  at  the  Haval  Air  Main'anance 
Training  Detachment  at  Patuxent  River,  Maryland,  in  a  manner 
similar  to  that  described  in  connection  with  the  first  group. 
In  this  instance,  however,  members  of  the  graduating  class 
were  paired  with  a  member  of  the  first  group  having  the  same 
or  a  similar  score  on  the  matching  variable. 

It  was  not  possible  to  match  perfectly  in  every  instance 
on  the  basis  of  grades  made  in  the  two  weeks  Aviation  Funda¬ 
mentals  Course,  as  had  been  true  in  earlier  phases  of  the 
study.  The  means  and  standard  deviations  of  the  two  groups 
were  quite  similar,  however.  The  means  were  80.52  and  80.41 
for  the  groups  receiving  30  weeks  and  22  weeks  training, 
respectively.  The  corresponding  standard  deviations  for  the 
two  groups  were  6.42  and  5.83.  Expressed  as  a  correlation 
coefficient,  the  correlation  between  pairs  on  the  matching 
variable  was  .95. 

Immediately  following  completion  of  the  Aviation  Elec¬ 
tronics  Technician  R  (Radar)  Course,  the  graduates  were  given 
two  weeks  leave,  at  the  end  of  which  they  reported  for  the 
course  at  the  Haval  Air  Maintenance  Training  Detachment, 
Patuxent  River,  Maryland. 

Two  primary  characteristics  were  sought  in  the  criterion 
measure.  First,  it  was  essential  that  the  equipment  course  be 
representative  of  the  equipment  maintenance  courses  taken  by 
graduates  of  the  Aviation  Electronics  Technician  R  (Radar) 
Course  in  terms  of  content  and  difficulty  level.  Second,  it 
was  necessary  that  performance  in  the  course  be  measured  as 


adequately  as  possible.  In  satisfying  the  first  of  these 
requirements  six  of  the  best  informed  personnel  at  the  head¬ 
quarters  of  the  Naval  Air  Maintenance  Training  Group  were 
asked  to  no«ninate  several  courses  that  they  considered  most 
representative  of  the  type  of  courses  that  v;ere  taken  by 
graduates  of  the  Aviation  Electronics  Technician  R  (Radar) 
Course,  This  action  was  taker,  to  reduce  the  large  number  of 
courses  offered  by  the  Naval  Air  Maintenance  Training  Group 
to  a  manageable  number. 

The  nominations  by  the  panel  of  six  judges,  first  per¬ 
forming  independently  and  later  as  a  panel  of  advisors,  pro¬ 
duced  a  list  of  ten  equipment  courses  that  were  considered 
most  representative  of  the  courses  taken  by  graduates  of  the 
Aviation  Electronics  Technician  R  (Radar)  Course.  Since 
assignment  to  equipment  courses  is  based  upon  the  equipment 
with  which  the  graduate  will  be  working  in  operating  squadrons, 
these  courses  tended  to  pertain  to  the  equipment  most  widely 
used  within  operating  squadrons.  This  list  of  ten  courses 
pertaining  to  specific  equipments  was  then  duplicated  and 
each  member  of  the  advisory  panel  was  asked  independently 
to  rank  the  courses  in  terns  of  their  representativeness. 

Three  courses  stood  out  as  the  most  representative,  and  a 
choice  was  made  among  these  three  on  the  basis  of  overall 
feasibility,  which  included  measurement  aspects,  and  location 
and  length  of  the  courses. 

Measurement  of  performance  in  the  cour-se  involved  a 
procedure  that  had  not  been  used  previously  in  training  re¬ 
search  within  the  Naval  Air  Technical  Training  Command.  The 
end  product  of  the  procedure,  and  the  primary  measure  of  per¬ 
formance  used  in  the  study,  was  a  ’’criterion  test,”  the  items 
of  which  corresponded  to  the  "s pecif ic-behavioral  objectives" 
of  the  course.  The  terms  criterion  test  and  specific  behavioral 
objectives  are  used  with  their  somewhat  specialized  connotation 
derived  from  programmed  instruction. 

In  developing  the  performance  measure,  the  senior  instruc¬ 
tional  programmer  for  the  Naval  Air  Maintenance  Training  Group, 
a  chief  petty  officer  with  many  years  of  electronics  experi¬ 
ence,  attended  the  entire  course  and  identified  the  specific 
behavioral  objectives  which  the  course  sought  to  achieve.  He 
then  constructed  one  or  more  "constructed  response"  items  (as 
contrasted  with  multiple  choice  type  items)  designed  to  mea¬ 
sure  each  of  the  specific  behavioral  objectives  of  the  course. 
The  objectives  and  test  items  were  discussed  with  the  personnel 
conducting  the  course  and  agreement  was  reached  as  to  their 
pertinence  and  inclusiveness. 
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Following  the  completion  of  each  of  the  ten  classes,  the 
test  was  scored  by  the  chief  petty  officer  who  had  constructed 
the  test.  The  test  contained  35  items,  and  the  score  assigned 
each  subject  was  the  number  of  items  answered  correctly  out 
Jvf  Of  the  35  items  on  the  test.  While  this  test  was  considered 
R  to  be  the  best  measure  of  performance  in  the  course,  two  other 
scores  also  were  collected.  These  were  scores  on  performance 
tests  of  troubleshooting  and  of  alignment  of  the  equipment. 
These  performance  tests  were  already  in  use  and  were  not 
altered  for  purposes  of  study. 

-  The  results  of  the  comparisons  that  were  made  between 
the  two  groups  in  terms  of  their  performance  in  a  represents** 

.  tive  operational  equipment  course  are  shown  in  Table  1.  A 
statistically  reliable  difference  was  not  found  between  the 
group  that  previously  had  received  the  original,  30  weeks 
course  and  the  group  that  had  received  the  revised,  22  weeks 
:‘!J-  course.  In  the  case  of  the  measure  that  was  considered  to 
be  the  best  measure  of  performance  in  the  specific  equipment 
course,  the  programmed  instruction  type  "criterion  test," 
the  mean  number  of  items  answered  correctly  on  the  35  item 
test  was  27.77  for  the  group  that  had  taken  the  original  course 
and  28.37  for  the  group  that  had  taken  the  revised  course. 


TABLE  1 


Comparison  of  the  Two  Hatched  Groups  in 
the  Operational  Equipment  Course 
(N  =  29  pai rs) 


Previous  Training 

"Cri ter  ion  Test" 

Al i qnment 

Troubleshooting 

Received 

Mean  S.  D. 

t 

Mean 

S.D,  t 

Mean 

S.D.  t 

Original  Course 
(30  weeks) 

27.77  3.88 

7^.72 

3.95 

73.90 

4.19 

•  Revised  Course 
(22  weeks) 

28.37  3.13 

.76 

76.59 

4.51  1.82 

75.90 

6.20  1.59 

Note;  A  t  value  of  2.01  is  required  for  significance  at  the  .05  level; 
2.67  at  the  .01  level . 


Similar  results  were  found  in  the  case  of  the  two  per¬ 
formance  measures  normally  used  in  the  course.  Hean  scores 
on  the  alignment  test  were  74.72  for  graduates  of  the  original 
course,  as  compared  with  76.59  for  graduates  of  the  revised 
course.  On  the  troubleshooting  measure  the  mean  scores  were 
73.90  and  75.90  for  graduates  of  the  original  and  revised 
course  respectively. 

There  are  several  points  that  appear  to  warrant  comment. 
The  first,  and  most  obvious,  is  that  the  study  gives  no  evi¬ 
dence  of  the  original,  longer  course  providing  better  prepa¬ 
ration  for  the  operational  equipment  course  than  the  revised, 
shorter  course  provides.  In  fact,  the  means  of  all  three 
measures  taken  upon  completion  of  the  equipment  course  were 
slightly,  but  not  reliably,  higher  for  graduates  of  the  re¬ 
vised  course  than  for  graduates  of  the  original  course. 

The  concept  of  using  performance  in  a  representative 
equipment  course  as  a  measure  of  the  adequacy  of  previously 
received  basic  training  was  applied  in  this  study  for  the 
first  time  in  naval  aviation  technical  training,  and  as  far 
as  the  investigators  are  aware,  anywhere.  On  the  basis  of 
experience  with  the  method,  it  appears  to  be  feasible  from 
the  standpoint  of  practical  application,  and  to  provide  in¬ 
formation  under  conditions  that  are  better  controlled  than 
ordinarily  is  possible  in  evaluation  studies  conducted  in 
the  fleet. 

While  the  methodological  aspects  of  the  study  doubtless 
are  of  primary  interest  to  this  conference,  in  your  capacity 
as  individual  taxpayers,  you  may  be  interested  to  know  that 
the  project  as  a  whole  resulted  in  something  over  20,000 
man-weeks  annually  being  available  for  productive  performance 
in  the  fleet  as  opposed  to  this  time  being  spent  in  a 
"consumer  status"  in  training. 
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EVALUATION  OF  A  PARTIALLY  SELF-PACED  COURSE 


Kirk  A.  Johnson 


It  is  generally  felt  that  education  or  training  is  most 
effective  when  it  is  tailored  to  the  characteristics  Ot 
abilities  of  the  individual  student*  The  cost  rudimentary 
form  of  adjustment  can  be  achieved  by  simple  variations  in 
training  time.  Even  this,  however,  has  been  relatively 
rare  in  military  training  situations.  The  adjustments  that 
are  made  in  situations  of  this  type  have  generally  taken  the 
form  of  night  schools  and  repetitions  for  students  who  are 
having  difficulty*  The  very  gifted,  highly  motivated  student 
has  occasionally  been  allowed  to  study  the  course  materials 
on  his  own.  In  recent  years,  two  or  three  track  training 
systems  have  been  developed  for  some  courses. 

The  most  obvious  obstacle  to  still  more  individualized 
forms  of  training  has  been  the  lack  of  efficient  means  for 
insuring  control  over  student  behavior.  Many  of  these  diffi¬ 
culties  have  been  reduced,  however,  through  recent  developments 
in  educational  techniques.  As  a  result,  a  second  obstacle 
has  become  much  more  prominent.  This  is  the  difficulty  in 
scheduling  instruction  in  such  a  way  that  the  student  can  be 
shifted  without  delay  between  portions  of  the  curriculun  that 
can  be  taught  most  efficiently  by  means  of  individualized 
instruction  and  other  portions  of  the  curriculum  in  which  in¬ 
dividualized  instruction  would  be  either  prohibitively  expen¬ 
sive  or  less  effective  than  alternative  means  of  instruction. 

In  certain  educational  situations  any  gaps  that  might  appear 
as  a  result  of  these  scheduling  difficulties  could  be  filled 
with  special  supplementary  instruction,  but  "enrichment"  of 
this  type  would  be  of  questionable  value  in  most  military 
training  situations.  As  a  result  of  these  difficulties,  the 
more  individualized  forms  of  instruction  have  been  used  in 
situations  that  preclude  a  realization  of  their  potential 
efficiency,  or,  in  relatively  fev;  cases,  in  courses  in  which 
the  entire  course  could  be  taught  by  means  of  such  techniques. 

I  would  like  to  present  a  brief  description  of  the  first 
individually  paced  course  to  be  developed  within  the  Naval 
Air  Technical  Training  Command,  The  course  was  the  Airborne 
Radio  Code  Operator  Course  in  which  the  student  learned  mili¬ 
tary  communications  procedures  and  how  to  send  and  receive 
International  Morse  Code,  During  the  first  part  of  the  course, 
which  lasted  about  one  week,  the  student  learned  the  symbols 
used  in  Morse  Code.  During  the  second  part,  which  lasted 
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about  three  weeks,  the  student  practiced  sending  and  re¬ 
ceiving  groups  of  random  alpha-numeric  symbols.  During  the 
third  and  final  part,  which  lasted  about  half  a  week,  the 
student  practiced  tending  aud  rtctii1  in*  irtusages  of  the 
type  that  he  would  encounter  ir  an  operational  military  en¬ 
vironment.  Military  Communication  Procedure*  was  taught  by 
means  of  30  lecture-discussion  sessions  distributed  over  the 
last  two  parts  of  the  course. 

There  were  several  reasons  for  selecting  this  course 
for  conversion  to  a  more  individual  format.  First,  the 
people  who  have  done  work  with  the  teaching  of  Morse  Code 
have  repeatedly  recommended  that  courses  of  this  type  be 
geared  to  the  progress  of  individual  students.  Of  equal 
importance  were  the  various  features  of  this  course  that 
made  such  a  conversion  somewhat  easier  than  it  might  have 
been  otherwise.  This  course  was  the  terminal  course  in  a 
sequence,  so  there  was  no  need  to  worry’ about  phasing  the 
students  into  another  course;  when  they  graduated  they  could 
be  sent  directly  to  their  assignments  in  the  field.  The 
practice  on  random  code  groups  was  already  being  handled  by 
means  of  tape  recorders,  so  a  major  portion  of  the  course 
could  be  presented  on  an  individualized  basis  without  further 
modifications.  Finally,  the  material  on  Military  Communica¬ 
tions  Procedures  lent  itself  readily  to  teaching  by  means  of 
programmed  instruction. 

In  the  individually  paced  version  of  the  course,  the 
first  week  of  the  course  was  taught  in  the  conventional 
manner.  During  the  next  phase  the  students  were  permitted 
to  progress  at  their  own  speed  through  practice  on  the  random 
code  groups  and  through  a  set  of  programmed  instruction  book¬ 
lets  that  covered  the  material  on  Military  Communications 
Procedures.  During  this  phase  the  instructor's  role  was 
limited  to  that  of  a  monitor.  As  soon  as  the  student  had 
reached  an  acceptable  level  of  proficiency  in  code  reception 
he  was  given  his  final  performance  test;  as  soon  as  he  had 
completed  the  programmed  instruction  booklets  he  was  given 
his  final  written  test.  If  he  passed  these  tests  he  was 
placed  on  orders.  He  graduated  immediately  prior  to  his 
departure  for  his  new  assignment.  The  interval  between  his 
passing  of  the  tests  and  his  graduation,  which  was  generally 
about  two  days  long,  was  spent  in  practice  on  sending  and 
receiving  military  messages. 

Evaluations  in  the  field  are  generally  beset  by  adminis¬ 
trative  problems,  but  in  the  present  case  these  problems  were 
more  serious  than  most.  During  the  preliminary  evaluation  of 


the  programmed  booklets  it  was  learned  that  the  course  was 
‘  being  transferred  from  I-ienpnis  in  an  effort  to  rectify  certain 
deficiencies  in  the  assignment  of  personnel  that  had  existed 
for  a  number  of  years.  V.’e  vert  able  to  postpone  this  trans¬ 
fer  until  we  could  gather  some  data  on  the  individually  paced 
course,  but  ouch  of  our  experimental  control  went  by  the  board. 

Data  were  gathered  on  88  entering  students  in  the  regu¬ 
lar  course  during  a  period  prior  to  the  completion  of  the  pro¬ 
grammed  instruction  booklets.  Data  were  also  gathered  on  66 
entering  students  during  a  period  in  which  the  programmed 
instruction  booklets  were  being  used  to  teach  Military  Commu¬ 
nications  Procedures.  There  was  no  individual  pacing  during 
this  period.  Finally,  data  were  gathered  on  28  entering 
students  in  the  individually  paced  version  of  the  course. 

The  three  groups  wei  evaluated  in  terms  of  attrition, 
class  days  per  graduating  student,  receiving  speed  after  15 
days,  and  scores  on  two  special  examinations.  Both  examina¬ 
tions  were  based  on  the  same  course  profile.  The  short  answer 
test  consisted  of  items  drawn  from  the  257  objectives  covered 
by  the  programs.  The  multiple  choice  test  consisted  of  items 
drawn  from  the  pool  used  in  the  construction  of  the  regular 
course  examinations. 

Scores  on  these  criteria  have  been  summarized  in  Table  1. 


TABLE  1 

Group  Means  on  Six  Criteria 
(Unmatched  Groups) 


Crl teria 

. 

Groups 

Requl ar 

Proqram 

Ind.  Paced 

%  Drops 

9.1 

9.1 

17.9 

%  Setbacks 

10.2 

16.7 

0.0 

Days  per  Graduate 

25.8 

24.0 

21.8 

Receiving  Speed 

9.1 

9.3 

9.3 

Short  Answer  Test  (%) 

77.3 

83.2 

82.1 

Multiple  Choice  Test  (%)  8 1 . 1 


79.0 


77.1 


The  total  percentage  of  drops  and  setbacks  was  less  in 
the  Individually  Paced  group  than  in  the  other  groups,  but 
the  percentage  of  drops,  which  is  probably  a  somewhat  more 
expensive  fore  of  attrition,  was  higher.  These  drops  tended 
to  oceur  quite  early  in  the  course,  at  a  point  when  none  of 
the  students  is  very  proficient  at  receiving  code  and  prior 
to  any  testing  on  Communications  Procedures.  Since  the  deci¬ 
sion  to  drop  a  student  is  based  on  fairly  subjective  considera¬ 
tions,  these  data  on  attrition  nay  not  provide,  a  very  reliable 
index  as  to  the  actual  value  of  the  various  courses.  Days 
per  graduate  was  computed  by  dividing  the  tine  spent  in  class 
by  all  entering  students  by  the  number  of  graduates,  so  these 
figures  reflect  differences  in  both  training  tine  and  attri¬ 
tion.  The  Conventional  course  r«  quired  about  26  days  per 
graduate.  This  figure  was  reduced  by  about  2  days  in  the 
Program  course,  and  by  2  more  days  in  the  Individually  Paced 
course.  The  two  groups  that  used  the  programs  were  quite 
similar  to  one  another  on  each  of  the  remaining  criteria. 

They  were  about  the  same  as  the  Regular  group  on  receiving 
speed,  about  5  percentage  points  better  on  the  short  answer 
test,  and  about  3  percentage  points  worse  on  the  multiple 
choice  test. 

These  data  were  fairly  encouraging  but  further  investiga¬ 
tions  indicated  that  students  entering  the  Individually  Paced 
course  tended  to  be  somewhat  better  than  students  entering 
the  other  courses  in  terns  of  both  general  aptitude  and  per¬ 
formance  in  previous  courses.  In  an  effort  to  compensate  for 
these  differences,  students  who  graduated  without  setbacks 
were  matched  on  the  basis  of  their  average  grade  in  the  three 
courses  preceding  the  course  under  investigation.  It  was 
possible  to  form  20  matched  triads.  The  data  from  these 
students  on  three  of  the  criteria  have  been  summarized  in 
Table  2. 

TABLE  2 


Group  Means  on  Three  Criteria 
(Matched  Groups) 


^  Criteria 

Group  Paced 

Regular 

Proqram 

Ind,  Paced 

^  Receiving  Speed 

9.2 

9.6 

9.1 

Short  Answer  Test  (%) 

VI 

77.5 

83.5 

83.1 

11  Multiple  Choice  Test  (%) 

82.5 

78.9 

78.0 

These  data  are  not  too  different  from  those  found  with 
the  unmatched  groups,  Receiving  speeds  arc  still  about  equal 
the-  differences  between  the  groups  using  programs  and  the  • 
regula  ••  group  have  decreased  slightly  on  the  short  answer 
test  a:.d  increased  slightly  on  the  multiple  choice  test. 


Although  inferential  statistics  are  not  particularly 
meaningful  in  situations  of  this  type,  it  was  found  that 
group  differences  on  the  short  answer  test  had  a  probability 
that  was  slightly  below  .05  and  that  group  differences  on 
the  multiple  choice  test  had  a  probability  that  was  slightly 
above  .05.  Differences  in  receiving  speed  were  far  from 
significant. 

In  summary,  the  Individually  Paced  Course  led  to  a 
reduction  of  about  16%  in  the  length  of  the  Regular  Course. 
The  primary. cost  was  a  drop  of  about  4.5  percentage  points 
on  the  multiple  choice  test,  but  even  this  is  probably  an 
overestimation,  since  for  the  Individually  Paced  students 
this  test  did  not  reflect  the  knowledge  they  might  have 
picked  up  during  their  last  two  days  of  practice  on  military 
messages.  The  9%  reduction  in  the  length  of  the  Program 
i course  could  be  viewed  as  pure  profit. 

Unfortunately  this  course  was  developed  by  circumventing 
rather  than  solving  most  of  the  problems  that  are  associated 
with  Individually  Paced  courses. 


-59 


CONSUMER  ACCEPTANCE  OF  PROGRAMMED  INSTRUCTION 


Howard  L.  Fleischman 


Along  with  determining  the  effectiveness  of  programmed 
instruction  and  investigating  new  methods  and  applications , 
it  may  be  desirable  to  assess  the  attitudes  of  consumers 
when  one  is  interested  in  making  the  most  effective  use  of 
programmed  instruction.  Since  programmed  instruction  is  a 
vital  part  of  the  training  effort  in  the  Naval  Air  Technical 
Training  Command,  a  good  opportunity  for  the  determination 
of  consumer  acceptance  was  available. 

A  questionnaire  was  assembled  consisting  of  28  questions 
and  statements  pertaining  to  programmed  instruction.  Sub¬ 
jects  were  asked  to  respond  to  the  questions  and  statements 
from  a  list  of  five  alternatives  which  ranged  from  the  very 
favorable  to  the  very  unfavorable.  In  order  to  obtain  answers 
which  were  as  honest  as  possible,  the  subjects  were  told  that 
the  results  would  be  used  for  research  purposes  only  and  that 
they  should  not  put  their  names  on  the  answer  sheets  as  this 
was  to  be  an  anonymous  questio  naire. 

The  subjects  consisted  of  three  types  of  consumers: 
students,  instructors,  and  training  administrators.  The 
students  were  Navy  and  Marine  enlisted  men,  for  the  most 
part  directly  out  of  recruit  training,  who  were  attending 
the  aviation  technical  training  schools  of  the  command.  The 
instructors  were  Navy  and  Marine  non-commissioned  officers. 

The  training  administrators  were  either  commissioned  officers 
or  civilian  education  specialists. 


The  participants  in  this  study  came  from  eight  schools 
of  the  Command.  These  schools  were: 

1.  Aviation  Electronics  Fundamentals  School  at 
Memphis,  Tennessee 

2.  Mechanical  Fundamentals  School  also  at  Memphis 

3.  Air  Controlman's  School  at  Glynco,  Georgia 

4.  Aviation  Ordnance  School  at  Jacksonville,  Florida 


5.  Aerographer • s  School  at  Lakehurst,  New  Jersey 


6 


Aviation  Boatswain1 s  Hate  School  also  at  Lakchurst 


7.  Aviation  Electrician's  School  at  Jacksonville ,  Florida 

8*  Photographer's  School  at  Pensacola,  Florida 

The  number  of  hours  or  programmed  instruction  in  these 
courses  ranged  from  55  in  Aviation  Electronics  Fundamentals 
tL'School  to  approximately  25  in  some  of  the  others.  The  same 
questions  were  presented  to  all  three  types  of  consumers 
tf-  except  for  a  few  where  the  wording  had  to  be  changed  to  make 
4  the  item  more  appropriate  to  the  particular  type  of  consumer. 

•  .Wp"' 

The  subjects  responded  to  the  items  on  a  scale  that 
%  ranged  from  favorable  to  unfavorable  opinions  of  programmed 
.instruction.  To  score  the  questionnaire,  the  alternative 
responses  were  credited  with  >i,  3,  2,  1  or  0  points  from  the 
J  favorable  to  the  unfavorable  end, respectively.  A  neutral  or 
>  uncommitted  response  to  a  question  was  scored  as  2  points, 
v  The  sum  of  the  item  credits  for  the  28  questions  represents 
1  the  subject's  total  score. 

Results  and  Discussion 

Table  1  shows  the  results  obtained  from  the  questionnaire 
for  each  sub-group.  The  maximum  or  most  favorable  score 
attainable  is  112,  with  56  being  the  neutral  response.  Taking 
a  look  at  the  individual  means,  the  student  groups  scored  sig¬ 
nificantly  above  the  neutral  point  in  each  of  the  schools  with 
the  Xviation  Electronics  Fundamentals  School  yielding  the 
highest  mean  (80.29). 

TABLE  1 

SUMMARY  DATA  BY  GROUPS 


Schools 


Av. 

Fund. 

Mech. 

Fund. 

Air 

Control . 

Av. 

Ord. 

Aeroq. 

Av. 

.  Boatswn. 

Av. 

Elect. 

Photo. 

Students 

X 

80.29 

76.35 

67.50 

68.69 

62.56 

77.21 

68.43 

67.89 

SD 

13.11 

15.49 

18.55 

19.26 

19.85 

16.70 

17.32 

18.79 

N 

128 

167 

126 

118 

111 

98 

165 

120 

P* 

.01 

.01 

.01 

.01 

.01 

.01 

.01 

.01 

Instructors 

X 

49.94 

54.33 

54.28 

68.96 

64.66 

80.57 

67.31 

51.55 

SD 

18.36 

19.13 

16.19 

15.52 

22.93 

10.15 

15.23 

21.21 

N 

51 

51 

14 

•  20 

3 

7 

60 

18 

P 

.05 

NS 

NS 

.01 

NS 

.01 

.01 

NS 

Administrators 

X 

51.84 

60.40 

67.14 

62.33 

62.50 

70.60 

69.11 

43.25 

SD 

18.14 

20.98 

10.49 

18.22 

18.49 

12.17 

23.32 

6.25 

N 

19 

10 

7 

9 

6 

5 

9 

4 

P 

NS 

NS 

.05 

NS 

NS 

.05 

NS 

.05 

*Probability  of  significant  difference  from  neutral  response. 


Three  cf  the  instructor  groups  scored  significantly 
above  the  neutral  point;  four  had  mean  scores  not  signifi¬ 
cantly  different  froi?.  the  neutral  point;  and  one  instructor 
group  (the  one  from  Aviation  Electronics  Fundamentals  School) 
scored  significantly  below  the  mean.  It  is  interesting  to 
note  that  the  school  where  the  instructors  scored  significantly 
below  the  mean  was  the  sane  school  whose  students  gave  the 
highest  approval  to  programmed  instruction. 

Going  on  to  the  training  administrator  groups,  we  find 
that  two  groups  scored  significantly  above  the  neutral  re¬ 
sponse;  five  administrator  groups  did  not  show  a  significant 
difference;  and  one  group  of  administrators  (the  Photographer's 
School  group)  scored  significantly  below  the  neutral  point. 

To  test  for  differences  between  the  groups,  an  analysis 
of  variance  for  groups  with  unequal  frequencies  was  computed. 

The  summary  table  for  this  analysis  is  shown  in  Table  2.  Dif¬ 
ferences  were  found  to  exist  between  types  of  consumers  at 
the  .01  level  of  significance  and  also  between  schools  at 
the  .01  level  of  significance.  No  interaction  effect  was  found. 

TABLE  2 


ANALYSIS  OF  VARIANCE  SUM  ARY  TABLE 


Source  of  Variance 

SS 

df 

HS 

F 

P 

Type  of  Consumer 

6.642.24 

2 

3,321.12 

10.72 

.01 

School 

10,567.20 

7 

1,509.60 

4.87 

.01 

Interaction 

820.46 

14 

58.60 

NS 

W?  thin 

403,068.00 

1302 

309.57 

Total 

421 ,097. SO 

1325 

317.80 

In  summarizing  the  results  obtained  from  the  questionnaire, 
we  can  say  that  the  students  look  upon  programmed  instruction 
very  favorably.  This  was  true  of  all  of  the  schools  that  were 
sampled.  Instructors  and  administrators  were  found  to  be  less 
favorable.  However,  in  only  one  school  each  did  the  instruc¬ 
tors  and  administrators  have  a  significant  negative  reaction 
towards  programmed  instruction. 


What  I  would  like  to  do  now  is  to  present  nine  individual 
questions  out  of  the  28  in  the  whole  survey  which  1  feel  might 
be  of  interest.  The  first  question  reads: 

"In  comparing  work  done  using  programmed  instruction 
with  studying  in  regular  textbooks,  I  feel  that  with  the 
same  amount  of  time  and  effort  I  learned: 

1*  Kuch  more  with  programs. 

2.  Somewhat  more  with  programs. 

3.  About  the  same. 

4.  Somewhat  more  from  textbooks. 

5.  Much  more  from  textbooks." 

The  results  showed  that  all  groups  averaged  above  the 
neutral  point,  that  is,  were  favorable  toward  programmed 
instruction.  An  analysis  of  variance  showed  that  significant 
differences  occurred  between  types  of  consumers  but  that  any 
differences  between  schools  were  not  significant. 

Another  question  asks: 

"Did  you  miss  not  being  able  to  participate  in  class 
discussions  or  miss  not  asking  questions  as  you  would  have 
in  a  conventional  class?" 

An  analysis  of  variance  showed  that  differences  exist 
between  types  of  consumers  but  not  between  schools,  ""he 
individual  group  means  for  most  instructor  and  administrator 
groups  fell  below  the  neutral  point  while  the  means  for  most 
student  groups  were  above  the  neutral  point.  Apparently,  the 
students  do  not  miss  class  discussion  as  much  as  the  instruc¬ 
tors  and  administrators  think  they  do. 

Another  question  reads: 

"How  do  you  rate  programmed  instruction  for  learning 
new  material?" 

The  analysis  of  variance  showed  that  all  differences  were 
significant,  between  types  of  consumers,  schools,  and  inter¬ 
actions.  All  groups  scored  above  the  neutral  paint  except 
the  administrators  at  the  Photographer 1 s  School,  Additionally 
the  students  were  consistently  more  favorable  toward  programmed 
instruction  than  instructors  or  administrators  except  for  the 
Aviation  Ordnance  School  where  the  students  and  instructors 
had  equal  means. 


"How  do 


A  companion  question  to  this  last  one  asks: 
you  rate  programs  for  reviewing  materials?" 

All  groups  scored  above  the  neutral  point  for  this 
question.  The  analysis  of  variance  showed  that  the  only 
differences  that  were  significant  were  those  between  schools. 

As  for  reviewing  material  from  the  programs,  at  the  end 
of  each  booklet  that  we  use,  there  is  a  test  which  the  stu¬ 
dent  takes  after  completing  the  prograra.  This  helps  the 
student  determine  the  areas  in  which  he  is  weak.  In  addition, 
about  half  of  these  tests  provide  the  students  with  page 
numbers  to  review  for  each  incorrect  answer.  These  tests 
are  considered  very  helpful  and  nay  contribute  to  the  ex¬ 
tremely  favorable  responses  on  this  question  by  the  instruc¬ 
tors  and  administrators. 

The  next  question  asks:  "To  what  extent  do  you  find 
the  pi’ograms  ‘repetitious? 

.  1.  Mot  at  all  repetitious. 

2.  Slightly  repetitious. 

3.  Uncertain. 

4.  Too  repetitious. 

5.  Much  too  repetitious.” 

A  companion  item  states:  ’'Programmed  instruction  is  a 
boring  method  of  learning."  The  subjects  were  asked  to  respond 

1.  Strongly  agree 

2.  Agree 

3.  Uncertain 

4.  Disagree 

5.  Strongly  disagree 

Analyses  of  variance  showed  significant  differences 
occurred  between  schools  for  both  items.  Two  student  groups 
fell  below  the  neutral  point  for  the  "repetitious"  question 
and  three  student  groups  fell  below  the  neutral  point  for  the 
"boring"  question.  These  results  may  be  interpreted  as  showing 
that  the  quality  of  the  programs  in  the  schools  vary  to  some 
extent  and  may  point  to  a  need  for  improving  this  aspect  of 
the  programs  at  some  of  the  schools. 

The  next  question  asks  about  retention  of  programmed 
materials  compared  to  retention  of  conventional  materials. 

Each  of  the  student  groups  scored  above  the  neutral  point, 
that  is,  were  more  favorable  toward  programmed  instruction. 
However,  the  instructors  and  administrators  at  over  half  of 
the  schools  felt  that  programmed  instruction  was  inferior  to 


conventional  instruction  in  this  regard.  An  analysis  of 
variance  showed  that  differences  in  attitudes  did  exist 
between  types  of  consumers  and  between  schools.  It  is  inte¬ 
resting  to  note  here  a  study  by  Dr.  Kirk  Johnson  which 
showed  that  students  did  ar.  well  or  better  on  a  final  exami¬ 
nation  after  taking  a  course  which  included  programmed  in¬ 
struction  than  after  taking  the  same  course  using  only  con¬ 
ventional  instruction* 

Another  statement  reads:  "Programmed  instruction  is  bad 
because  there  is  never  a  teacher  around  to  explain  anything." 

The  subjects  are  again  asked  to  respond: 

1*  Strongly  agree 

2*  Agree 

3*  Uncertain 

4*  Disagree 

5.  Strongly  disagree 

An  analysis  of  variance  showed  that  significant  differences 
occurred  between  types  of  consumers  on  this  item.  From  the 
individual  means,  it  appears  that  the  instructors  feel  that 
they  are  available  to  the  student  more  often  than  the  students 
feel  they  are.  This  is  an  area  that  will  have  to  be  checked 
more  closely. 

Finally,  the  next  item  was  used  as  a  summary  question. 

It  asks:  "In  terms  of  an  overall  evaluation  of  programmed 
instruction  at  this  training  activity,  I  believe  that  pro¬ 
grammed  instruction: 

1.  Is  making  (or  will  make)  a  major  contribution. 

2.  Is  making  (or  will  make)  a  contribution  of  some 

importance . 

3.  Is  no  better  or  worse  than  what  we  have  had  before. 

4.  Is  (or  will  be)  somewhat  detrimental. 

5.  Is  (or  will  be)  very  detrimental." 

Although  the  analysis  of  variance  showed  that  differences 
between  types  of  consumers  and  interaction  differences  did 
w-cur,  the  individual  group  means  for  all  groups  fell  above 
the  neutral  point.  It  is  clear  that  all  consumers  felt  that 
programmed  instruction  is  making  a  worthwhile  contribution 
to  the  training  program  in  the  Naval  Air  Technical  Training 
Command . 
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training,  and  programmed  learning. > 
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